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EXECUTIVE  SUMMARY 


PROBLEM 

In  recent  years,  virtual  reality  (VR),  often  called  virtual  environments  (VE),  has  received  an 
enormous  amount  of  attention  among  training  developers.  This  is  due  in  part  to  media  hyperbole 
over  applications  of  VR  in  the  entertainment  industry,  but  even  more  so,  because  training 
developers  recognize  the  potential  of  VR  as  a  flexible  and  effective  training  medium.  Training 
effectiveness  evaluations  (TEEs)  of  VR  training  systems  can  help  define  the  appropriate  use  of 
VR  technologies  to  achieve  this  potential. 

A  prime  candidate  for  examining  the  effectiveness  of  VR  systems  is  the  training  of  ship 
handling  for  the  surfaced  submarine.  Although  land-based  simulator  facilities  currOTtly  exist  for 
training  Submarine  Piloting  and  Navigation  teams,  these  systems  do  not  provide  detailed  harbor 
and  channel  ship  handling  training  for  the  Officer  of  the  Deck  (OOD).  OOD  training  is  primarily 
obtained  firom  on-the-job  experience,  which  is  adversely  impacted  by  the  operational  constraints  of 
the  Submarine  Force,  and  the  limited  surfaced  steaming  time  of  submarines.  Training  that  will 
expose  junior  officers  (JOs)  to  a  variety  of  geographical  and  environmental  conditions  is  very 
limited  since  most  Commanding  Officers  place  their  most  experienced  OODs  on  watch  during 
these  challenging  evolutions.  Therefore,  an  alternative  simulation-based  training  capability  is 
needed.  A  VR-based  simulation  may  provide  this  necessary  capability  if  it  is  shown  to  be  an 
effective  training  approach. 

OBJECTIVE 

The  objective  of  the  Virtual  Environment  for  Submarine  OOD  Ship  Handling  Training 
(VESUB)  project  was  threefold:  (1)  to  develop,  demonstrate,  and  evaluate  the  trainmg  potential  of 
a  stand-alone  virtual  reahty-based  system  for  OOD  training;  (2)  to  determine  if  this  system  can  be 
integrated  with  existing  Submarine  Piloting  and  Navigation  (SPAN)  training  simulators,  and  (3)  to 
determine  the  viability  of  virtual  reality  technology  as  a  training  tool.  This  report  documents  a  two- 
phased  TEE  of  die  VESUB  technology  demonstration  system  at  Submarine  Training  Facility 
(SUBTRAFAC),  Norfolk,  VA  and  the  Naval  Submarine  School  (NAVSUBSCOL),  Groton,  CT. 
The  results  of  the  TEE,  as  documented  in  this  report,  will  be  used  in  conjunction  with  the  Instructor 
Station  usability  analysis  (documented  in  a  separate  publication;  Hays,  Seamon,  &  Bradley,  1997) 
to  help  ensure  that  the  design  of  the  follow-on  operational  VESUB  system  can  incorporate  the  latest 
technologies,  be  as  user-fiiendly  as  possible,  and  apply  the  most  effective  training  methods  and 
techniques. 

APPROACH 

A  two-phased  TEE  was  conducted  using  Navy  personnel  as  subjects.  The  first  phase  was 
conducted  during  a  three  week  period  at  SUBTRAFAC,  Norfolk,  VA  and  the  second  during 
another  three  week  period  at  NAVSUBSCOL,  Groton,  CT.  At  each  site,  the  VESUB  system  was 
set  up  in  a  room  where  the  TEE  could  be  conducted  without  mterfering  with  ongoing  training  at 
the  training  conomands.  Forty-one  evaluation  participants  experienced  three  VESUB  scenarios: 
the  first,  was  an  orientation  scenario  to  allow  them  to  experience  and  practice  system 
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functionality;  the  second,  was  a  training  scenario  which  targeted  several  specific  ship  handling 
tasks  (derived  from  a  perceptual  and  cognitive  task  analysis);  and  the  third,  was  a  comparable 
scenario  to  test  the  trainees’  improvement  on  these  ship  handling  skills.  Prior  to  the  first 
scenario,  demographic  data  were  collected  and  a  “comfort”  questionnaire  was  administered  to 
assess  the  participants’  physical  condition.  After  completion  of  each  scenario,  a  post-experience 
“comfort”  questionnaire  (with  identical  elements)  assessed  any  physical  changes  experienced  by 
the  participants.  Finally,  each  participant  provided  an  assessment  of  the  VESUB  system  and 
recommendations  for  its  use  and  improvement. 

RESULTS 

Data  were  collected  on  fifteen  ship  handling  variables  grouped  into  seven  skill  categories.  A 
mixed  factorial  analysis  of  variance  (ANOVA)  design,  with  experience  as  the  between-subjects 
variable  and  scenario  session  (training  and  testing)  as  the  within  subjects  variable,  found 
significant  learning  (skill  improvements)  for  all  experience  levels  (0  to  14  years)  on  eleven  of  the 
fifteen  variables.  For  example,  trainees  improved: 

♦  39%  in  checking  range  markers 

♦  33%  in  visually  checking  the  rudder 

♦  13%  in  issuing  correct  turning  commands 

♦  57%  in  contact  management  skills 

♦  44%  in  reaction  time  during  a  man  overboard  (MOB)  event 

♦  29%  in  using  correct  commands  during  the  MOB  event 

♦  40%  in  using  correct  commands  during  a  yellow  soimding  event 

No  major  simulator  side  effects  problems  were  found  in  the  TEE,  even  though  trainees  averaged 
almost  two  hours  in  the  head-mounted  display.  Details  on  these  and  other  data  are  presented  in 
the  report,  as  are  the  results  of  opinion  questionnaires  administered  to  both  trainees  and 
observers. 

CONCLUSIONS  AND  RECOMMENDATIONS 

The  results  of  the  TEE  strongly  demonstrate  that  the  VESUB  technologies  can  provide 
effective  training  on  ship  handling  skills  for  both  novices  and  experts.  It  is  recommended  that 
VESUB  move  forward  toward  the  procurement  of  an  operational  system.  It  is  also 
recommended  that  an  event-based  training  curriculiun  be  developed  to  train  the  perceptual  and 
cognitive  skills  required  for  ship  handling. 
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INTRODUCTION 


PROBLEM 

In  recent  years,  virtual  reality  (VR),  sometimes  called  virtual  environments  (VE),  has  received 
an  enormous  amount  of  attention  among  training  developers.  This  is  due  in  part  to  media 
hyperbole  over  apphcations  of  VR  in  the  entertainment  industry,  but  even  more  so,  because 
training  developers  recognize  the  potential  of  VR  as  a  flexible  and  effective  training  medium. 
Training  effectiveness  evaluations  of  VR  training  systems  can  demonstrate  the  effectiveness  of 
VR  technologies  and  also,  how  to  optimally  use  them  in  existing  and  future  training  programs. 
The  Virtual  Environment  for  Submarine  OOD  Ship  Handling  Training  (VESUB)  technology 
demonstration  system  provided  an  opportunity  to  evaluate  the  training  effectiveness  of  a  VR- 
based  training  system  in  an  operational  context,  using  personnel  like  those  who  will  eventually 
train  with  the  operational  (follow-on)  VESUB  system  and  other  future  VR-based  training 
systems. 

OBJECTIVE 

This  is  the  second  in  a  series  of  reports  to  document  the  results  of  the  VESUB  project.  It 
provides  the  results  of  a  two-phased  training  effectiveness  evaluation  (TEE)  of  the  VESUB 
technology  demonstration  system  at  two  Navy  training  facilities:  the  Submarine  Training 
Facility  (SUBTRAFAC),  Norfolk,  VA;  and  the  Naval  Submarine  School  (NAVSUBSCOL), 
Groton,  CT.  This  report  will  be  used  in  conjunction  with  a  previous  publication  (Hays,  Seamon, 
and  Bradley,  1997),  which  documented  the  usability  analysis  of  the  VESUB  Instructor/Operator 
Station  (lOS),  in  the  development  of  the  specifications  for  the  operational  VESUB  system  and  to 
support  recommendations  for  how  to  optimally  use  these  VR  training  technologies.  A  third 
report  in  this  series  will  document  the  lessons  learned  from  the  project.  It  is  hoped  that  these 
reports  will  provide  guidance  to  increase  the  training  effectiveness  of  future  VR  training  systems. 

ORGANIZATION  OF  THE  REPORT 

The  Background  Section  provides:  1)  a  description  of  the  submarine  ship  handling  task  and 
the  training  requirement  for  die  VESUB  training  system;  2)  a  brief  overview  of  VR  training 
research  and  an  explanation  of  why  submarine  ship  handling  was  chosen  as  the  technology 
demonstration  task.;  and  3)  a  short  description  of  the  VESUB  project.  A  functional  description 
of  VESUB  is  included  to  provide  a  context  for  the  reader.  Next,  is  an  overview  of  the  perceptual 
and  cognitive  task  analysis  that  guided  the  hardware,  software,  and  instructional  capabilities  of 
the  system.  Short  descriptions  of  the  developmental  phases  of  the  project  and  the  scenario 
construction  approach  complete  the  Background  Section.  The  Method  Section  describes  the 
subjects,  equipment,  materials,  and  procedures  used  in  the  TEE  and  the  Results  Section  provides 
analyses  of  the  data  collected  during  the  TEE.  The  Discussion  Section  provides  in-depth 
explanations  of  the  data  and  their  importance  for  ship  handling  training.  Finally,  the  implications 
of  the  data  and  how  they  can  guide  the  development  of  the  operational  VESUB  system  as  well 
as,  other  VR-based  training  systems  are  discussed  in  the  Conclusions  and  Recommendations 
Section. 
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THE  VIRTUAL  ENVIRONMENT  FOR  SUBMARINE  OOD  SHIP  HANDLING 
TRAINING  TECHNOLOGY  DEMONSTRATION  SYSTEM  (VESUB) 

BACKGROUND 

Land-based  simulator  facilities  currently  exist  for  training  Submarine  Piloting  and  Navigation 
teams.  These  systems  do  not,  however,  provide  detailed  harbor  and  channel  ship  handling  training 
for  the  Officer  of  the  Deck  (OOD).  OOD  training  is  primarily  obtained  through  on-the-job 
experience,  which  is  adversely  impacted  by  the  operational  constraints  of  the  Submarine  Force, 
and  the  limited  surfaced  steaming  time  of  submarines.  Training  of  junior  officers  (JOs)  under  a 
variety  of  geographical  and  environmental  conditions  is  very  limited  since  most  Commanding 
Officers  place  their  most  experienced  OOD  on  watch  during  these  challenging  evolutions. 
Therefore,  an  alternative,  simulation-based  training  capability  is  needed  and  may  be  met  effectively 
and  economically  using  virtual  reality  (VR)  technologies. 

Research  in  Virtual  Reality 

VR  has  received  considerable  publicity  in  recent  years  jfrom  the  entertainment  industry  (e.g.. 
Brill,  1994;  Gradecki,  1994)  and  the  popular  press  (e.g.,  Rheingold,  1991;  Kalansky,  1993;  Burdea 
&  Coiffet,  1994).  VR  has  also  generated  considerable  interest  on  the  part  of  the  military  (e.g., 
McVey,  1993;  Cook,  1994;  Larapton,  Knerr,  Goldberg,  Bliss,  Moshell,  &  Blau,  1994;  Knerr, 
Goldberg,  Lampton,  Witmer,  Bliss,  Moshell,  &  Blau,  1994)  and  NASA  (e.g..  Null  &  Jenkins, 
1993). 

Traditional  military  training  systems  have  used  operational  equipment,  large  training  ranges,  or 
expensive  simulation  equipment.  VR  affords  the  potential  to  greatly  reduce  the  cost  of  training 
systems  because  it  can  provide  a  trainee  with  an  interface  to  training  equipment  without  the 
necessity  of  replicating  expensive  hardware.  Furthermore,  VR  training  can  be  provided  on  a  range 
of  tasks  using  the  same  basic  trainee  interface  by  changing  the  VR  software.  ITus  can  potentially 
save  significant  amounts  of  resomces  which  would  otherwise  be  required  to  build  a  completely 
new,  physically  different,  trainee  interface.  VR  also  offers  the  potential  to  provide  new  and 
innovative  training  strategies  that  can  only  be  applied  in  a  virtual  world.  For  example,  a  trainee 
could  view  the  inside  of  a  fuel  system  from  the  perspective  of  a  molecule  of  gasoline;  he  or  she 
could  grow  to  the  size  of  a  giant  to  view  a  tactical  engagemait  from  above,  then  jump  from  one 
ship  to  another  for  various  points  of  view;  or  a  trainee  could  engage  in  team  training  with  virtual 
team  members.  However,  the  potential  of  VR  training  can  not  be  achieved  unless  researchers 
explore  the  complex  nature  of  this  training  medium  and  demonstrate,  evaluate,  and  enhance  the 
effectiveness  of  VR  training  systems. 

Research  in  VR  is  being  conducted  at  various  laboratories  (e.g.,  the  Army  Research  Institute, 
and  the  Navy  Post  Graduate  School)  in  three  modality  areas:  visual,  auditory,  and  haptic.  Visual 
research  includes  efforts  to  improve  display  devices  (e.g.,  head-mounted  displays  [HMDs]  and 
“cave”  projection  screens)  and  head  tracking  systems.  The  visual  modality  is  the  most  highly 
advanced  area  of  VR  research.  Auditory  research  is  examining  the  effectiveness  of  the  use  of 
highly  realistic  and  3-D  localized  sounds  in  virtual  worlds.  Research  on  haptic  interfaces  seeks  to 
improve  the  representation  of  botii  the  tactile  (cutaneous)  and  kinesthetic  (muscle  and  joint)  aspects 
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of  virtual  touch.  Because  the  haptic  area  is  the  least  mature  of  the  three  VR  modalities,  a  task  was 
sought  that  did  not  require  a  haptic  interface.  As  discussed  below,  submarine  ship  handling  was 
chosen  as  the  task  area  for  this  VR  technology  demonstration. 

The  VESUB  Project 

VESUB  was  a  Navy  Manpower,  Personnel,  and  Training  Research  and  Development  project 
with  the  goal  to  develop,  demonstrate,  and  evaluate  the  training  effectiveness  of  a  VR-based  system 
for  surfaced  submarine  OOD  training.  The  submarine  ship  handling  task  was  chosen  as  the  task 
area  for  the  technology  demonstration  for  three  reasons:  1)  there  is  a  clear  Navy  need  for  this 
training;  2)  submarine  ship  handling  is  a  visually  dependent  task;  and  3)  the  visual  modality  is  the 
most  mature  of  the  three  areas  of  VR  development.  If  effective,  VESUB  has  the  potential  to  reduce 
the  number  of  ship  handing  mishaps,  save  lives,  and  reduce  property  loss.  Based  on  the  results  of 
the  VESUB  effort,  the  Navy  will  procure  several  operational  VESUB  systems  to  be  installed  at 
submarine  training  facilities  throughout  the  United  States. 

The  software  and  the  hardware  integration  for  the  VESUB  system  was  developed  by  Nichols 
Advanced  Marine  Engineering  (formerly  Advanced  Marine  Enterprises,  [AME])  under  contract  to 
and  direction  of  a  research  team  at  the  Naval  Air  Warfare  Center  Training  Systems  Division 
(NAWCTSD)  in  Orlando,  Florida.  The  research  team  included:  Research  Psychologists,  Visual 
Engineers,  Computer  Engineers,  and  submarine  Subject  Matter  Experts  (SMEs).  To  save  time  and 
money,  VESUB  was  developed  by  leveraging  AME’s  Virtual  Ship,  a  commercial  product  that  has 
been  used  for  several  years  to  train  surface  ship  handling  tasks. 

Under  the  VESUB  project,  AME  has  modified  Virtual  Ship  by  replacing  its  large  projection 
screen  system  with  head-moimted  display  (HMD)  and  magnetic  head  tracking  technologies  to 
present  the  visual  scene  to  trainees.  ITie  VESUB  project  has  also  required  AME  to  enhance  the 
level  of  detail  and  fidelity  in  the  visual  databases  far  beyond  that  of  previous  systems  and  also  to 
incorporate  a  voice  recognition  system  for  trainee  interaction  with  the  system.  In  addition  to 
advancements  in  the  hardware  and  software  technologies  used  in  Virtual  Ship,  VESUB  has 
introduced  niunerous  improvements  to  the  instructor  interface  so  it  can  be  more  easily  used  by 
Navy  instructors. 

VESUB  SYSTEM  FUNCTIONAL  DESCRIPTION 

The  VESUB  technology  demonstration  system  uses  a  high-resolution  HMD  to  provide  the 
trainee  with  a  simulated  360  degree  visual  environment  containing  many  of  the  required  cues 
associated  with  harbor  and  chaimel  navigation  as  well  as,  accurate  cultural  features  and  varying 
environmental  conditions.  Speaker-independent  voice  recognition  and  speech  synthesis  are  used  as 
the  trainee  interface  with  the  computer,  providing  communications  training  for  ^s  complex  task. 
Visual  scene  rendering,  computation  of  harbor  currents  and  wind  effects,  and  the  hydrodynamic 
properties  of  own  ship  and  otiier  traffic  ships  are  realistically  represented.  Appendix  A  lists  and 
describes  the  hardware  and  software  used  in  VESUB. 
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Figure  1  shows  an  artist’s  depiction  of  the  VESUB  system.  On  the  right  side  of  the  figure,  an 
instructor  is  shown  seated  in  jfront  of  the  three  monitors  at  the  Instructor/Operator  Station  (lOS). 
Screens  on  two  of  the  lOS  monitors  are  used  to  modify  and  control  training  scenarios.  The  third 
monitor  is  used  to  observe  the  virtual  world  that  the  trainee  sees  through  the  HMD  during  a  training 
exercise.  This  view  allows  the  instructor  to  evaluate  the  trainee’s  performance  in  real-time  and 
provide 


Figure  1:  Artist’s  Representation  of  VESUB 

guidance  to  direct  the  trainee  to  look  for  objects  that  support  the  task.  This  same  monitor  is  used 
before  an  exercise  to  create  tiie  training  scenarios.  In  Figure  1,  the  trainee  is  shown  standing  in  the 
bridge  mock-up,  wearing  the  HMD  and  communicating  with  a  simulated  navigation  team  via  a 
hand-held  microphone.  The  inset  depicts  a  typical  harbor  scene  as  the  trainee  views  it  through  the 
HMD.  The  visual  scene  displays  distant  objects,  including:  harbor  geographic  features,  navigation 
aids,  landmarks,  and  traffic  ships.  It  also  displays  objects  near  the  trainee,  mcluding:  a 
representation  of  the  bridge  area  (for  either  the  6881  or  726  class  of  submarines),  and  the  bridge 
equipment  and  information  used  to  help  the  OOD  perform  the  ship  handling  task.  This  includes: 
the  bridge  suitcase  with  the  rudder  angle  indicator  and  engine  order  telegraph;  the  compass 
repeater;  a  representation  of  a  simplified  harbor  chart;  and  a  course  card  with  information  on  each 
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leg  of  the  channel.  The  requirement  to  display  near  and  distant  objects  necessitated  the  use  of  new 
visual  scene  management  processes  (e.g.,  a  large  area  database  management  [LADBM]  system) 
that  were  not  needed  in  previous  ship  handling  training  systems.  When  the  trainee  turns  his  head,  a 
magnetic  head  tracker,  mounted  above  the  mock-up,  senses  the  movement  and  the  computer 
changes  the  visual  scene  appropriately.  For  example,  the  trainee  is  able  to  turn  to  the  stem  and 
observe  the  mdder  move  in  response  to  an  issued  helm  order.  Visual  obstacles,  such  as  lookouts, 
periscopes,  and  radar  masts  can  be  displayed  at  the  discretion  of  the  instmctor  to  make  the  training 
task  more  reahstic  by  forcing  the  trainee  to  look  around  them  to  see  critical  visual  cues. 

PERCEPTUAL  AND  COGNITIVE  ANALYSIS  OF  THE  SUBMARINE  SHIP 
HANDLING  TASK 

A  submarine,  when  surfaced,  must  perform  according  to  the  ship  handling  procedures 
followed  by  surface  ships.  Current  submarine  OOD  ship  handling  training,  primarily  conducted 
on-the-job,  requires  each  ship’s  Commanding  Officer  (CO)  to  evaluate  the  performance  of  JOs 
prior  to  certifying  them  as  qualified  OODs.  In  the  early  phases  of  the  VESUB  project,  submarine 
COs  were  interviewed  to  determine  how  they  determined  a  JO  was  qualified  as  OOD.  In  almost 
every  case,  the  COs  responded  that  their  personnel  were  qualified  when  they  had  developed  the 
“Seaman’s  Eye.”  This  concept  was,  therefore,  used  as  the  keystone  for  the  analysis  of  the 
training  task  that  the  VESUB  technology  demonstration  system  had  to  support. 

Initially,  the  VESUB  research  team  viewed  the  ship  handling  task  as  primarily  cognitive; 
requiring  the  OOD  to  make  timely  decisions  about  the  ship’s  condition  and  issue  orders  to  ensure 
its  safety.  However,  the  use  of  the  term  “Seaman’s  Eye”  by  task  experts  shifted  the  emphasis  to 
perceptual  issues.  Therefore,  it  was  reasoned  that  the  ship  handling  task  had  to  be  composed  of 
both  perceptual  and  cognitive  components.  Prior  to  additional  focused  interviews  with  SMEs, 
the  following  definition  of  “Seaman’s  Eye”  was  developed  to  guide  the  perceptual  and  cognitive 
analysis  of  the  ship  handling  task. 

Seaman ’s  Eve:  The  total  situation  awareness  of  the  ship  handling 

environment  and  the  ability  to  safely  maneuver  the  vessel  in  all  conditions. 

“Seaman’s  Eye”  includes  situation  awareness  as  an  integral  component.  Situation  awareness 
has  been  defined  as  “the  perception  of  the  elements  in  the  environment  within  a  volume  of  time 
and  space,  the  comprehension  of  their  meaning,  and  the  projection  of  their  status  in  the  near 
future”  (Endsley,  1988,  p.  97).  This  implies  that  in  order  to  develop  good  situation  awareness 
(e.g.,  “Seaman’s  Eye”),  the  individual  must  develop  a  mental  model  of  his  own  ship,  other  ships, 
the  environment,  and  the  task  (e.g.,  safely  make  a  turn).  Rouse  and  Morris  (1986)  define  a 
mental  model  as  a  “mechanism  whereby  humans  generate  descriptions  of  system  purpose  and 
form,  explanations  of  system  functioning  and  observed  system  states,  and  predictions  of  future 
system  states”  (p.  360).  To  construct  a  mental  model  that  is  accurate  and  complete,  the 
individual  must  perceive  the  information  available  in  the  environment  (perception),  and  process 
the  information  to  assess  the  current  status  of  the  environment  and  project  the  future  status  of  the 
environment  (cognition).  Then,  the  mental  model  can  be  used  to  help  the  individual  take 
appropriate  actions  to  efficiently  and  effectively  perform  the  required  tasks. 
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The  quality  of  an  individual’s  situation  awareness  is  directly  dependent  upon  the  degree  of 
completeness  and  accuracy  of  the  mental  model  that  has  been  developed.  The  more  accurate  the 
mental  model,  the  better  the  individual  will  be  able  to  process  critical  information  in  an  efficient 
and  effective  manner.  Task  performance  follows  a  similar  progression;  a  more  accurate  and 
complete  mental  model  of  the  environment  and  task  should  lead  to  improved  task  performance. 
The  goal  of  any  training  system  (including  VESUB)  should  be  to  enhance  the  individual’s 
mental  model  of  the  task  situation. 

With  this  goal  in  mind,  the  definition  of  “Seaman’s  Eye”  was  used  to  guide  the  development 
of  hardware,  software,  and  instructional  approaches  for  VESUB.  However,  the  definition  was 
still  too  general  to  meet  this  goal.  Therefore,  iterative  probing  of  SMEs,  using  focused  group 
discussions  at  Navy  training  facilities  and  in  the  NAWCTSD  laboratory,  detailed  the  perceptual 
and  cognitive  components  that  make  up  this  complex  concept.  Additional  group  discussions 
with  the  SMEs  helped  the  VESUB  research  team  to  organize  the  components  as  shown  in  Table 
1. 


Table  1 

Components  of  “Seaman’s  Eye” 


PERCEPTUAL  COMPONENTS 


IP.  Locating  and  Identifying  Navigation  Aids 


2P.  Judging  Distance 


3P.  Identifying  the  Start  and  Completion  of  Turns 


4P.  Locating,  Identifying,  and  Avoiding  Obstacles 


5P.  Sense  of  Ship’s  Responsiveness 


6P.  Recognizing  Environmental  Conditions 


TP.  Recognizing  Equipment  Failures 


8P.  Detecting  and  Filtering  Communications 


COGNITIVE  COMPONENTS 


1C.  Understanding  the  Relationship  of  Visual  Cues  to  their  Representations  on  Charts 


2C.  Understanding  Relative  Size  and  Height/Range  Relationships,  and 
Angle  on  the  Bow  (AOB 


3C.  Understanding  Advance  and  Transfer 


4C.  Understanding  the  Effects  of  Tides,  Currents,  Wind,  and  Seas _ 


5C.  Understanding  Rules  of  the  Road  _ 


6C.  Understanding  Relative  Motion  (Direction  and  Speed 


7C.  Understanding  Methods  to  Differentiate  and  Prioritize  Traffic  Contacts 


8C.  Understanding  Ship’s  Operation  Under  Harbor  Directives 


9C.  Understanding  Methods  to  Deal  with  Uncooperative  Traffic 


IOC.  Understanding  Correct  Operation  of  Ship’s  Systems  _ 


lie.  Understanding  When  and  How  to  Take  Corrective  Actions 


12C.  Understanding  Effective  Communication  Procedures  _ 
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The  perceptual  components  of  “Seaman’s  Eye”  were  initially  used  to  determine  the  hardware 
requirements  for  the  system.  For  example,  the  requirement  to  locate  and  identify  navigation  aids 
(IP)  and  to  judge  distance  (2P)  required  that  the  trainee  be  able  to  see  the  objects  at  great 
distances  necessitated  the  use  of  a  high  resolution  HMD.  The  requirement  to  visually  inspect  the 
rudder  necessitated  the  use  of  head  tracking  and  rapid  refresh  rate  of  the  visual  scene.  Details  on 
this  determination  ean  be  found  in  Hays,  Castillo,  Bradley,  and  Seamon  (1997). 

Both  the  perceptual  and  the  cognitive  components  of  “Seaman’s  Eye”  were  used  to  develop 
and  organize  the  training  objectives  for  the  VESUB  system.  These  training  objectives,  shown  in 
Appendix  B,  were  subsequently  used  to  develop  the  training  and  evaluation  criteria  for  the 
VESUB  TEE. 

VESUB  DEVELOPMENT  PHASES 

The  VESUB  project  proceeded  in  three  phases:  requirements  determination,  formative 
evaluations,  and  training  effectiveness  evaluations.  A  major  tool  used  during  the  requirements 
determination  phase  was  a  feasibility  demonstration  system  developed  under  the  NAWCTSD 
exploratory  research  Virtual  Environment  Training  Technology  project.  The  feasibility 
demonstration  system  included  a  simplified  harbor  scene  and  submarine  model,  which  was 
viewed  through  a  low  resolution  HMD.  This  system  afforded  SMEs  a  chance  to  experience  a 
virtual  world.  Questionnaires  and  interviews  were  then  used  to  solieit  detailed  functional 
requirements  for  the  VESUB  technology  demonstration  system  from  the  SMEs.  These 
requirements  were  documented  in  a  NAWCTSD  special  report  (Tenney,  Briscoe,  Pew,  Bradley, 
Seamon,  &  Hays,  1996)  and  used  in  conjunction  with  the  work  statement  to  direct  the  contractor 
during  the  development  of  the  VESUB  technology  demonstration  system. 

The  formative  evaluation  phase  was  conducted  in  the  laboratory  at  NAWCTSD.  Whenever 
AME  delivered  an  improved  iteration  of  the  VESUB  software,  it  was  evaluated  against  the 
functional  requirements  by  the  VESUB  researeh  team.  Data  for  the  formative  evaluations  were 
also  collected  from  eleven  fleet  and  school  SMEs  and  nine  Navy  reservists  with  ship  handling 
experienee  following  extensive  exposure  to  VESUB.  As  soon  as  the  formative  data  were  collected, 
the  results  were  provided  to  the  software  developers  to  guide  system  improvements.  The  formative 
evaluations  focused  on  both  the  funetionality  of  the  trainee  interface  (e.g.,  the  fidelity  of  objects  in 
the  visual  seene  or  the  fiinctionality  of  the  voice  recognition  system)  and  the  usability  of  the  lOS 
(Hays,  et  al.,  1997). 

The  TEE  of  the  VESUB  technology  demonstration  system  was  conducted  at  the  Submarine 
Training  Facility  in  Norfolk,  Virginia  and  the  Naval  Submarine  School  in  Groton,  Connecticut 
during  the  Winter  and  Spring  of  1998.  The  TEE  used  Navy  trainees  with  various  levels  of 
experience  (novice  to  expert)  to  determine  the  effectiveness  of  the  VESUB  system  and  also  to  help 
determine  how  the  technology  can  be  integrated  into  Navy  training. 
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TEE  SCENARIO  CONSTRUCTION 

The  development  of  the  training  and  testing  scenarios  used  in  the  TEE  followed  an  “evrait- 
based”  training  approach  (Johnston,  Cannon-Bowers,  &  Smith-Jentsch,  1995;  Dwyer,  Fowlkes, 
Oser,  &  Lane,  1997).  “The  cornerstone  of  event-based  training  is  a  process-based  performance 
measurement  system  that  is  linked  closely  to  training  objectives  that  are  embedded  in  pre-specified 
scenario  events”  (Johnston,  et  al.,  1995,  p.  275).  This  approach  has  proven  successful  for  training 
tasks  in  a  variety  of  areas,  including;  shipboard  command  teams  (Johnston,  et  al.,  1995);  aviation 
team  coordination  (Fowlkes,  Lane,  Salas,  Franz,  &  Oser,  1994);  multi-service  team  training, 
distributed  among  multiple  remote  sites  (Dwyer,  et  al.,  1997).  A  similar  approach  was  followed  in 
the  development  and  construction  of  the  scenarios  used  in  the  TEE. 

The  cognitive  and  perceptual  components  of  “Seaman’s  Eye”  (Table  1)  and  the  training 
objectives  derived  from  them  (Appendix  B)  were  used  to  develop  the  ship  handling  skill  training 
events  for  the  TEE  scenarios.  Ei^t  major  events  were  created  for  the  training  and  testing 
scenarios: 

♦  Position  Determination 

♦  Contact  Location  and  Evaluation 

♦  Getting  Underway 

♦  Maneuvering  Own  Ship  (e.g.,  correct  turns) 

♦  Man  Overboard 

♦  Incorrect  Navigator  Report  (position) 

♦  Crossing  Situation  (rules  of  the  road) 

♦  Yellow  Soimding 

Details  on  the  variables  used  to  measure  performance  during  these  events  and  the  perceptual  and 
cognitive  components  that  support  them  are  discussed  in  later  sections  of  this  report.  The  limited 
time  and  resources  available  for  the  TEE  constrained  the  number  of  ship  handling  skills  that  could 
be  trained  and  evaluated.  Nevertheless,  this  event-based  approach  ensured  that  the  TEE  scenarios 
were  both  instructionally  sound  and  task  relevant. 
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METHOD 

The  two-phased  VESUB  TEE  was  conducted  at  the  SUBTRAFAC,  Norfolk,  Virginia  and  the 
NAVSUBSCOL,  Groton,  Connecticut.  Three  weeks  were  spent  at  each  site,  including  system 
set  up  and  testing,  data  collection,  and  demonstrations  for  senior  officers  and  other  visitors. 

SUBJECTS 

Forty-one  active  duty  Navy  personnel  participated  as  trainees  in  the  TEE.  They  included: 

♦  4  Naval  Officers  firom  the  staff  at  SUBTRAFAC,  Norfolk,  VA 

♦  1  Chief  Petty  Officer  firom  the  staff  at  SUBTRAFAC,  Norfolk 

♦  13  Naval  Officers  firom  submarine  crews  in  Norfolk 

♦  1  Senior  Chief  Petty  Officer  fi*om  the  staff  at  Submarine  Force  Atlantic 

♦  6  Officers  firom  the  staff  at  NAVSUBSCOL,  Groton,  CT 

♦  6  students  firom  the  Submarine  Officers  Basic  Course  (SOBC),  Groton 

♦  10  Junior  Officers  firom  submarine  crews  in  Groton 

Demographic  information  on  the  trainees  is  provided  in  Appendix  C. 

A  Norfolk  harbor  pilot  (civilian  government  employee)  was  run  through  the  scenarios,  but 
because  of  his  lack  of  familiarity  with  submarine-specific  procedures,  only  his  side  effects  and 
opinion  data  are  reported.  Eleven  additional  persons  participated  in  the  TEE  as  observers. 
Demographics  on  all  these  observers  are  provided  in  Appendix  D. 

EQUIPMENT 

The  equipment  used  for  the  TEEs  consisted  of  the  VESUB  system  (see  Appendix  A)  and  two 
hand-held  stopwatches. 

MATERIALS 

A  scenario  script  was  used  to  ensure  all  participants  received  identical  information  (see 
Appendix  I  for  the  inbound-first  script).  Performance  data  were  collected  using  the  forms  shoAvn 
in  Appendix  J.  In  addition,  trainees  and  observers  filled  out  the  following  questiormaires: 

♦  A  trainee  demographic  and  background  information  questioimaire  (see  Appendix  E) 

♦  A  “comfort”  questioimaire,  administered  four  times,  to  assess  the  subject’s  feelings  on  16 
physical  symptoms  (see  Appendix  F).  The  “comfort”  questionnaires  were  adapted  fi'om 
the  Simulator  Sickness  Questionnaire  (SSQ),  based  on  data  fi*om  a  factor  analysis  of  over 
1000  Navy  and  Marine  Corps  pilots  in  fli^t  simulators  (Lane  &  Keimedy,  1988; 
Keimedy,  Lane,  Berbaum,  &.  Lilienthal,  1993).  This  approach  has  also  been 
recommended  to  evaluate  side  effects  fi'om  virtual  environments  (Kennedy,  Lane, 
Lilienthal,  Berbaum,  &  Hettinger,  1992) 

♦  A  questioimaire  soliciting  trainee’  opinions  about  the  system  (shown  in  Appendix  G) 

♦  An  Observer  Questionnaire  (shown  in  Appendix  H) 
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PROCEDURE 

At  each  TEE  site,  the  VESUB  evaluation  was  conducted  in  a  dedicated  room  to  avoid 
interference  with  or  interruptions  from  other  training  classes.  The  research  team  included  an 
“instructor”  who  operated  Ae  system  during  the  evaluation  scenarios,  two  data  collectors;  and 
one  evaluation  coordinator,  whose  job  was  to  provide  introductory  briefings  for  the  overall 
evaluation  and  for  each  scenario  and  to  ensure  that  all  procedures  were  conducted  in  the  same 
manner  for  each  subject.  The  TEE  scenarios  were  run  according  to  a  script  to  ensure  that  all 
subjects  received  identical  instructions.  Subjects  were  randomly  coimterbalanced  so  that  half 
received  training  on  the  inbound  scenario  and  testing  on  the  outbound  scenario  and  half 
experienced  scenarios  in  the  opposite  order. 

Two  subjects  were  evaluated  each  day,  one  in  the  morning  and  one  in  the  afternoon.  The  total 
time  for  each  session  (including  breaks)  was  approximately  three  hours.  Upon  arrival,  each 
subject  filled  out  the  demographic  questionnaire  (Appendix  C)  and  the  first  “comfort” 
questionnaire  (Appendix  F).  Each  subject  then  experienced  three  scenarios.  The  first  scenario, 
using  the  Norfolk  database,  provided  orientation  to  the  system,  demonstrations  of  system 
capabilities,  and  an  opportunity  for  the  subject  to  learn  to  adjust  the  HMD  and  to  move  about  in 
the  virtual  world.  He  also  was  given  an  opportunity  to  practice  interactions  with  the  voice 
recognition  system  to:  1)  learn  to  commimicate  with  the  virtual  Helm,  Navigator,  and 
Maneuvering  Watch;  2)  to  learn  to  control  the  binocular  function,  the  harbor  chart,  and  the 
course  card.  The  mean  duration  of  the  orientation  scenario  was  22  minutes  and  7  seconds. 
Immediately  following  the  orientation  scenario,  each  subject  filled  out  the  second  “comfort” 
questionnaire. 

After  a  short  break,  each  subject  was  given  a  prebrief  for  the  training  scenario  (Appendix  I), 
then  placed  in  the  bridge  mock-up  and  fitted  into  the  HMD.  The  training  scenario  used  the  Kings 
Bay,  Georgia  database  and  was  either  inboimd  or  outbound,  based  on  random  coimterbalancing. 
The  training  scenario  targeted  several  specific  ship  handling  tasks,  as  described  earlier.  At  the 
close  of  each  training  event,  the  system  was  paused  and  an  instructional  intervention  was 
provided.  These  interventions  followed  the  basic  script  shown  in  Appendix  I,  with  modifications 
based  on  each  subject’s  performance  on  the  event.  The  mean  duration  of  the  training  scenario 
was  51  minutes.  At  the  end  of  the  training  scenario,  each  subject  was  debriefed  using  the 
performance  data  collected  by  the  research  team  (Appendix  J)  and  several  printouts  provided  by 
the  system.  For  example,  the  system  produced  a  track  plot,  which  showed  the  subject  where  his 
ship  was  located  relative  to  the  center  of  the  channel  and  to  various  traffic  ships  throughout  the 
scenario.  The  debrief  also  included  a  replay  of  the  scenario  at  ten  times  normal  speed  with  the 
subject’s  eye  point  positioned  above  and  behind  the  submarine  so  the  subject  could  see  an 
overview  of  the  channel  and  traffic  patterns  relative  to  own  ship’s  actions.  Following  this 
debrief,  each  subject  took  a  ten  minute  break. 

After  the  break,  each  subject  received  a  prebrief  on  the  third  (testing)  scenario.  This  scenario 
was  the  reverse  (inbound  or  outbound)  of  the  training  scenario.  During  the  testing  scenario,  the 
subject  was  evaluated  on  comparable  tasks  to  those  that  were  trained  in  the  previous  scenario. 

The  mean  duration  of  the  testing  scenario  was  37  minutes.  Although  no  training  interventions 
were  provided  in  this  scenario,  each  subject  was  fully  debriefed  on  his  performance  after  he  had 
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filled  out  the  final  “comfort”  questionnaire  and  the  questionnaire  on  his  opinions  about  system 
features  and  possible  future  uses  of  the  technology  (see  Appendix  G).  Finally,  each  subject  was 
thanked  for  his  participation  and  asked  not  to  discuss  the  specifics  of  the  TEE  scenarios  with 
other  prospective  subjects. 
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RESULTS 


SIMULATION  SIDE  EFFECTS 

The  raw  data  on  simulation  side  effects,  collected  with  the  “comfort”  questionnaire,  are 
shown  in  Appendix  K.  Using  the  methods  developed  by  Kennedy,  et  al.  (1993),  the  raw  scores 
were  transformed  into  three  subfactor  scores  (Nausea,  Oculomotor,  and  Disorientation)  and  a 
Total  Side  Effects  Score.  These  transformed  scores  and  the  duration  each  subject  was  in  the 
HMD  can  also  be  found  in  Appendix  K.  The  frequency  of  the  Total  Side  Effects  Scores  for  the 
Pretest  and  after  each  scenario  are  shown  in  Table  2  and  Figure  2. 


Table  2 


Frequency  of  Total  Side  Effects  Scores  During  the  VESUB  TEE 


Total  SSQ  Score 
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Figure  2: 

Frequency  of  Total  Simulator  Side  Effects  Scores 
During  the  VESUB  TEE 
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The  mean  values  on  each  SSQ  subfactor  scale  are  shown  in  Figure  3. 


Session 


Figure  3: 

Mean  Scores  on  SSQ  Subfactors  During  the  VESUB  TEE 


SfflP  HANDLING  PERFORMANCE 

The  trainees’  improvement  on  several  ship  handling  skills  was  assessed  by  comparing  their 
performance  on  selected  tasks  in  the  training  scenario  to  their  performance  on  comparable  tasks 
in  the  testing  scenario.  Performance  data  were  collected  on  fifteen  dependent  variables  grouped 
into  seven  skill  areas.  These  dependent  variables  and  skill  areas  are  listed  and  defined  in  Table  3 
The  perceptual  and  cognitive  components  of  “Seaman’s  Eye”  (see  Table  1)  that  are  most  closely 
related  to  each  variable  are  also  shown  in  Table  3  (the  most  relevant  components  are  underlined 
and  in  bold  face). 

A  mixed  factorial  analysis  of  variance  (ANOVA)  design  was  used  to  analyze  the  data 
collected  on  the  variables.  The  between-subjects  factor  was  years  of  experience  (less  than  one 
year,  one  to  three  years,  and  over  three  years).  The  within-subjects  factor  was  scenario  session 
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(training  or  testing).  To  control  for  the  repeated  ANOVAs,  a  significance  level  of  .01  was  set  for 
the  analysis.  The  results  of  the  analysis  are  reported  according  to  the  skill  groupings  fi-om  Table 
3. 


Table  3: 

Dependent  Variables  Used  in  the  VESUB  TEE 


Variable 

Explanation  of  Variable 

Related  “Seaman’s 
Eye”  Components 

1  Position  Determination 

Position  of  Own 
Ship  Across  Track 

Difference  between  the  actual  starting 
position  (right  or  left  of  track)  and  the 
reported  position,  as  determined  by  use 
of  navigation  aids  and  range  markers. 

IP,  2P,  1C,  2C 

Position  of  Own 
Ship  Along  Track 

IE,  2P,  1C,  2C 

1  Contact  Management 

Contacts  Found 

Ratio  of  the  total  number  of  contacts 
foimd  to  the  total  number  of  contacts 
available. 

2P,  4P,  2C 

Contacts  “Of 
Concern” 

Ratio  of  the  total  number  of  contacts 
identified  as  being  of  concern  to  the 
total  number  actually  of  concern,  as 
determined  by  SMEs. 

2P,  4P,  2C,  5C,  6C,  7C 

General  Ship  Handling 

Turning  Commands 

Ratio  of  the  number  of  correct 
commands  given  by  subject  when 
making  turns  to  the  total  possible 
number  of  correct  turning  commands 
for  all  turns  during  the  entire  scenario. 

IP,  2P,3P,5P,  1C, 
3C,4C,  IOC,  12C 

Checking  the 
Rudder 

Ratio  of  the  number  of  times  the 
subject  visually  checked  the  rudder 
after  giving  a  rudder  order  to  the  total 
number  of  possible  visual  checks  after 
rudder  orders  during  the  entire 
scenario. 

IOC 

Checking  Ranges 

Ratio  of  the  number  of  times  the 
subject  visually  checked  the  range 
markers  to  the  total  possible  number  of 
times  ranges  should  have  been  checked 
during  the  entire  scenario,  as 
determined  by  SMEs. 

IE,  IOC 

Table  Continued  on  the  Next  Page  | 
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Table  3:  Continued 


Variable  Name 

Explanation  of  Variable 

‘‘Seaman’s  Eye” 
Components  Related  To 

1  Emergettcy  OperatioBS  ^ 

Man  Overboard 
Reaction  Time 

Time  to  begin  corrective  actions  after 
hearing  report  that  a  man  was  overboard. 

8P,  6C.  lie.  12C 

Man  Overboard 
Commands 

Number  of  correct  commands  given  and 
actions  performed  during  MOB  event. 

6C,  nc,  12C 

Yellow  Soimding 

Total  number  of  correct  actions  taken  by 
the  subject  during  Yellow  Sounding  event. 

IP,  2P,  8P,  4C,  IOC, 
lie.  12C 

1'.:^''  .  .  lBCori^et.iL€port-Ree%0itioB<'-^-^ 

Incorrect  Report 
Recognition 

Total  number  of  correct  actions  taken  and 
commands  issued  by  the  subject  when 
given  an  incorrect  position  report  by  the 
Navigator. 

IP,  2P,  8P,  nc,  12C 

Commands  to  Get 
Underway 

Ratio  of  number  of  correct  commands 
given  when  getting  own  ship  underway  to 
the  total  possible  number  of  correct 
commands  to  get  own  ship  underway 
during  the  entire  scenario. 

5P,  4C,  IOC,  12C 

Acknowledge  Reports 

Ratio  of  the  number  of  scheduled  reports 
that  are  acknowledged  by  the  subject  to  the 
total  possible  number  of  required 
acknowledgments 

8P.  1QG.12C 

Use  Station  Identifiers 

Number  of  times  the  trainee  failed  to  use 
station  identifiers  when  issuing  commands. 

IOC,  12C 

Ferry  Passage 

The  total  number  of  correct  commands 
given  during  the  crossing  situation  with  the 
ferry. 

2P,  4P,  5P,  8P,  2C,  SC, 
6C,  7C,8c,ioc,  nc, 

12C 
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Position  Determination 

Two  variables  were  used  to  assess  the  trainees  learning  of  position  determination:  position 
across  the  track  and  position  along  the  track.  Position  across  the  track  was  the  difference 
between  the  trainee’s  assessment  of  own  ship’s  position  and  its  actual  position.  A  mixed 
factorial  ANOVA  yielded  a  significant  main  effect  for  scenario  session,  F(l,  36)  =  7.71,  p  = 
.009).  The  mean  score  on  this  variable  for  the  training  scenario  was  34.07  yards  and  for  the 
testing  scenario  was  13.65  yards.  No  significant  main  effect  was  foimd  for  the  between-subjects 
factor  of  experience  level,  F(2, 36)  =  1.05,  p  =  .360  and  no  significance  was  found  for  the 
interaction  of  scenario  session  and  experience  level,  F(2,  36)  =  .77,  p  =  .471,  indicating  that 
learning  occurred  across  all  levels  of  experience.  Figure  4  shows  a  graphic  representation  of 
trainees’  performance  on  determination  of  position  across  the  track  during  the  training  and 
testing  scenarios. 


Figure  4: 

Trainees’  Performance  on  Determination  of  Position  Across  the  Track 

During  the  VESUB  TEE 

The  trainees’  performance  on  the  determination  of  own  ship’s  position  along  the  track  was 
rated  as  satisfactory  if  a  trainee’s  estimate  was  within  200  yards  (approximately  one  Trident 
submarine’s  length)  of  the  actual  position  of  own  ship.  A  mixed  factorial  ANOVA  yielded  no 
significant  main  effect  for  scenario  session,  F(l,  36)  =  1.96,  p  =  .171,  no  significant  main  effect 
for  experience  level,  F(2,  36)  =  .396,  p  =  .676,  and  no  significant  effect  for  the  interaction  of 
scenario  session  and  experience  level,  F(2,  36)  =  .743,  p  =  .483. 
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Contact  Management 

Two  variables  were  used  to  assess  the  trainees’  performance  in  the  contact  management  skill 
area:  Contacts  Found  and  Contacts  of  Concern.  Contacts  found  was  a  ratio  of  the  total  number 
of  contacts  found  by  the  trainee  when  scanning  the  virtual  world  compared  to  the  actual  number 
of  contacts  present.  A  mixed  factorial  ANOVA  yielded  no  significant  main  effect  for  scenario 
session,  F(l,  36)  =  .280,  p  =  .600.  No  significant  main  effect  was  found  for  the  between-subjects 
factor  of  experience  level,  F(2, 36)  =  .645,  p  =  .531. 

A  significant  interaction,  however,  was  found  between  scenario  session  and  experience  level, 
F(2, 36)  =  5.59,  p  =  .008.  Figure  5  shows  a  graphic  representation  of  the  performance  of  trainees 
on  the  task  of  finding  contacts  during  the  training  and  testing  scenarios. 
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Figure  5: 

Trainees’  Performance  on  Locating  Contacts  During  the  VESUB  TEE 

The  means  for  the  trainees  with  “less  than  1  year”  of  ship  handling  experience  were  .75  and 
.91  for  the  training  and  testing  scenarios  respectively.  The  means  for  the  trainees  with  “1  to  3 
years”  of  ship  handling  experience  were  .85  and  .89  for  the  training  and  testing  scenarios 
respectively.  The  ratio  of  contacts  foimd  compared  to  actual  contacts  increased  for  these  two 
experience  levels  between  training  and  testing  indicating  that  learning  did  occur.  The  means  for 
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the  trainees  with  “more  than  3  years”  of  ship  handling  experience  were  .93  and  .79  for  the 
training  and  testing  scenarios  respectively.  The  ratio  of  contacts  found  compared  to  actual 
contacts  decreased  for  this  experience  level  indicating  that  learning  did  not  occur  for  this  group 
on  this  particular  task.  Possible  reasons  for  this  decline  in  performance  in  this  group  will  be 
discussed  later  in  this  report. 

The  trainees’  performance  for  contact  management  with  regards  to  determination  of  which 
contacts  should  be  categorized  as  contacts  “of  concern”  was  a  ratio  measure  of  the  number  of 
contacts  found  that  the  trainee  categorized  as  “of  concern”  compared  to  the  number  of  contacts  in 
the  scenario  that  were  categorized  as  “of  concern”  by  SMEs  employed  in  development  of  the 
training  and  testing  scenarios.  A  mixed  factorial  ANOVA  yielded  a  significant  main  effect  for 
scenario  session,  F(l,  36)  =  28.02,  p  =  .000.  The  mean  score  on  this  variable  for  the  training 
scenario  was  .49  and  for  the  testing  scenario  was  .77.  This  indicates  that  out  of  100%  of  the 
contacts  the  trainee  should  have  identified  as  “of  concern”  in  the  training  and  testing  scenarios, 
the  trainee  correctly  identified  49%  in  the  training  scenario  and  77%  in  the  testing  scenario.  No 
significant  main  effect  was  found  for  the  between-subjects  factor  of  experience  level,  F(2,  36)  = 
1.35,  p  =  .271  and  no  significance  was  found  for  the  interaction  of  scenario  session  and 
experience  level,  F(2,  36)  =  .689,  p  =  .509.  Figure  6  shows  a  graphic  representation  of  the 
performance  of  trainees  on  the  task  of  identification  of  contacts  “of  concern”  during  the  training 
and  testing  scenarios. 


Figure  6: 

Trainees’  Performance  on  Identification  of  Contacts  of  Concern  During  the  VESUB  TEE 
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General  Ship  Handling 

Three  variables  were  used  to  assess  the  trainees  learning  of  general  ship  handling:  turning 
commands,  checking  the  rudder  and  checking  ranges.  Turning  commands  was  the  ratio  of  the 
number  of  commands  issued  and/or  actions  taken  associated  with  turning  own  ship  that  the 
trainees  executed,  during  the  entire  scenario,  compared  with  the  number  of  commands  issued 
and/or  actions  taken  associated  with  turning  own  ship  that  the  trainees  should  have  executed  as 
determined  by  SMEs.  A  mixed  factorial  ANOVA  5delded  a  significant  main  effect  for  scenario 
session,  F(l,  36)  =  17.36, 2  =  -000.  The  mean  score  on  this  variable  for  the  training  scenario  was 
.75,  and  for  the  testing  scenario  was  .85.  This  indicates  that  out  of  100%  of  the  commands  and 
actions  associated  with  turning  own  ship  that  the  trainees  should  have  executed  in  the  training 
and  testing.scenarios,  the  trainees  correctly  executed  75%  in  the  training  scenario  and  85%  in  the 
testing  scenario.  Figure  7  shows  a  graphic  representation  of  the  performance  of  trainees  on 
execution  of  turning  commands  during  the  training  and  testing  scenarios. 
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Figure  7: 

Trainees’  Performance  on  Turning  Commands  During  the  VESUB  TEE 

No  significant  main  effect  was  found  for  the  between-subjects  factor  of  experience  level,  F(2, 
36)  =  3.90, 2  =  .03,  and  no  significance  was  found  for  the  interaction  of  scenario  session  and 
experience  level,  F(2, 36)  =  1 .54,  p  =  .228,  indicating  that  learning  was  not  affected  by  level  of 
experience  and  that  learning  occurred  across  all  levels  of  experience. 
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Visually  checking  the  rudder  was  the  ratio  of  the  number  of  times  the  trainees  physically 
turned  around  to  visually  check  the  rudder  compared  with  the  number  of  times  the  trainees 
should  have  visually  checked  the  rudder  as  determined  by  SMEs.  A  mixed  factorial  ANOVA 
yielded  a  significant  main  effect  for  scenario  session,  F(l,  36)  =  17.29,  p  =  .000.  The  mean  score 
on  this  variable  for  the  training  scenario  was  .54,  and  for  the  testing  scenario  was  .72.  This 
indicates  that  out  of  100%  of  the  possible  times  that  the  trainees  should  have  physically  turned 
around  to  visually  check  the  rudder  in  the  training  and  testing  scenarios,  the  trainees  correctly 
checked  the  rudder  54%  of  the  time  in  the  training  scenario  and  72%  of  the  time  in  the  testing 
scenario.  Figure  8  shows  a  graphic  representation  of  the  performance  of  trainees  on  visually 
checking  the  rudder  during  the  training  and  testing  scenarios. 


Figure  8: 

Trainees’  Performance  on  Visually  Checking  the  Rudder  During  the  VESUB  TEE 

No  significant  main  effect  was  fovmd  for  the  between-subjects  factor  of  experience  level,  F(2, 
36)  =  .898,  g  =  .416,  and  no  significance  was  found  for  the  interaction  of  scenario  session  and 
experience  level,  F(2,  36)  =  2.75,  g  =  .077,  indicating  that  learning  was  not  affected  by  level  of 
experience  and  that  learning  occurred  across  all  levels  of  experience. 
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Visually  checking  ranges  was  the  ratio  of  the  number  of  times  the  trainees  visually  checked 
the  ranges  compared  with  the  number  of  times  die  trainees  should  have  visually  checked  the 
ranges  as  determined  by  SMEs.  A  mixed  factorial  ANOVA  yielded  a  significant  main  effect  for 
scenario  session,  F(l,  36)  =  26.68,  p  =  .000.  The  mean  score  on  this  variable  for  the  training 
scenario  was  .54  and  for  the  testing  scenario  was  .75.  This  indicates  that  out  of  100%  of  the 
possible  times  that  the  trainees  should  have  visually  checked  the  ranges  in  the  training  and 
testing  scenarios,  the  trainees  correctly  checked  the  ranges  54%  of  the  time  in  the  training 
scenario  and  75%  of  the  time  in  the  testing  scenario.  Figure  9  shows  a  graphic  representation  of 
the  performance  of  trainees  on  visually  checking  the  ranges  during  the  training  and  testing 
scenarios. 
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Figure  9: 

Trainees’  Performance  on  Visually  Checking  Ranges  During  the  VESUB  TEE 

No  significant  main  effect  was  found  for  the  between-subjects  factor  of  experience  level,  F(2, 
36)  =  1.36,  p  =  .271.  A  significant  effect  was  found  for  the  interaction  of  scenario  session  and 
experience  level,  F(2, 36)  =  5.28,  p  =  .010.  Figure  10  shows  a  graphic  representation  of  this 
interaction. 
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Figure  10: 

Interaction  of  Experience  Level  and  Range  Checking  Performance 
During  the  VESUB  TEE 

The  means  for  the  trainees  with  “less  than  1  year”  of  ship  handling  experience  were  .40  and 
.79  for  the  training  and  testing  scenarios  respectively.  The  means  for  the  trainees  with  “1  to  3 
years”  of  ship  handling  experience  were  .56  and  .75  for  the  training  and  testing  scenarios 
respectively.  The  means  for  the  trainees  with  “more  than  3  years”  of  ship  handling  experience 
were  .65  and  .70  for  the  training  and  testing  scenarios  respectively.  The  greatest  amount  of 
improvement  occurred  in  the  “less  than  1  year”  ship  handling  experience  group.  The  “1  to  3 
years”  ship  handling  experience  group  improved  less  than  the  “less  than  1  year”  group  but  more 
than  the  “more  than  3  year”  group.  Tlie  least  amoimt  of  improvement  occurred  in  the  “more  than 
3  years”  ship  handling  experience  group.  This  does  not  mean,  however,  that  learning  did  not 
occur  across  all  three  experience  levels.  This  simply  illustrates  the  point  that  the  most 
experienced  group  performed  at  an  initially  higher  level  in  the  training  scenario  than  the  other 
two  groups,  and  as  a  result  of  this  initially  higher  level  of  performance,  improved  the  least 
between  training  and  testing. 
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Emergency  Procedures 

Three  variables  were  used  to  assess  the  trainees  learning  of  appropriate  actions  to  be  taken 
during  emergency  situations:  man  overboard  reaction  time,  man  overboard  commands  and 
yellow  soxmding.  Man  overboard  reaction  time  was  the  time  between  the  occurrence  of  the  man 
overboard  event  and  the  time  until  the  trainee  initiated  the  first  correct  command  and/or  took  an 
action  associated  with  proper  handling  of  the  emergency  event.  A  mixed  factorial  ANOVA 
yielded  a  significant  main  effect  for  scenario  session,  F(l,  36)  =  17.99,  p  =  .000,  indicating  that  a 
significant  improvement  in  reaction  time  occurred.  The  mean  score  on  this  variable  for  the 
training  scenario  was  5.34  seconds,  and  for  the  testing  scenario  was  2.99  seconds.  The  trainees’ 
overall  improvement  in  terms  of  time  to  react  to  this  emergency  was  2.35  seconds.  Figure  1 1 
shows  a  graphic  representation  of  the  performance  of  trainees  on  reaction  time  during  the 
training  and  testing  scenarios  for  the  man  overboard  emergency  event. 
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Figure  11: 

Trainees’  Performance  on  Man  Overboard  Reaction  Time  During  the  VESUB  TEE 

No  significant  main  effect  was  found  for  the  between-subjects  factor  of  experience  level,  F(2, 
36)  =  .525,  p  =  .596,  and  no  significance  was  found  for  the  interaction  of  scenario  session  and 
experience  level,  F(2, 36)  =  .084,  p  =  .920,  indicating  that  learning  was  not  affected  by  level  of 
experience  and  that  learning  occurred  across  all  levels  of  experience. 

Man  overboard  commands  was  the  number  of  correct  commands  issued  in  response  to  the 
man  overboard  event.  A  mixed  factorial  ANOVA  yielded  a  significant  main  effect  for  scenario 
session,  F(l,  36)  =  21.96,  p  =  .000,  indicating  that  a  significant  improvement  in  the  number  of 
correct  commands  and/or  actions  taken  during  the  man  overboard  emergency  event  had  occurred. 
The  mean  score  on  this  variable  for  the  training  scenario  was  3.74  commands  issued,  and  for  the 
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testing  scenario  was  5.25  commands  issued.  Figure  12  shows  a  graphic  representation  of  the 
performance  of  trainees  on  number  of  correct  commands  and/or  actions  taken  for  the  man 
overboard  emergency  event  during  the  training  and  testing  scenarios. 
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Figure  12: 

Trainees’  Performance  on  Man  Overboard  Commands  During  the  VESUB  TEE 

No  significant  main  effect  was  found  for  the  between-subjects  factor  of  experience  level,  F(2, 
36)  =  .821,  p  =  .448,  and  no  significance  was  found  for  the  interaction  of  scenario  session  and 
experience  level,  jF(2,  36)  =  2.21,  p  =  .125,  indicating  that  learning  was  not  affected  by  level  of 
experience  and  that  learning  occurred  across  all  levels  of  experience. 

Yellow  soimding  was  the  number  of  correct  commands  issued  in  response  to  a  yellow 
sounding  emergency  situation.  A  mixed  factorial  ANOVA  yielded  a  significant  main  effect  for 
scenario  session,  F(l,  36)  =  14.76,  p  =  .000,  indicating  that  a  significant  improvement  in  the 
number  of  correct  commands  and/or  actions  taken  during  the  yellow  sounding  emergency  event 
had  occurred.  The  mean  score  on  this  variable  for  the  training  scenario  was  3.74  commands 
issued,  and  for  the  testing  scenario  was  5.25  commands  issued.  Figure  13  shows  a  graphic 
representation  of  the  performance  of  trainees  on  handling  of  the  yellow  sounding  emergency 
event  during  the  training  and  testing  scenarios. 
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Figure  13: 

Trainees’  Performance  on  Yellow  Sounding  Actions  and  Commands 

During  the  VESUB  TEE 

No  significant  main  effect  was  found  for  the  between-subjects  factor  of  experience  level,  F(2, 
36)  =  1.96,  p  =  .155,  and  no  significance  was  found  for  the  interaction  of  scenario  session  and 
experience  level,  F(2, 36)  =  .352,  p  =  .706,  indicating  that  learning  was  not  affected  by  level  of 
experience  and  that  learning  occurred  across  all  levels  of  experience. 

Incorrect  Report  Recognition 

Incorrect  Position  Determination  was  the  number  of  correct  commands  issued  or  actions  taken 
in  response  to  an  incorrect  position  rq)orted  by  own  ship’s  navigator.  Examples  of  correct 
actions  include:  checking  ranges  or  other  navigation  aids,  asking  for  a  repeat  of  the  navigator’s 
report,  or  disagreeing  with  the  report.  A  mixed  factorial  ANOVA  yielded  a  significant  main 
effect  for  scenario  session,  F(l,  36)  =  7.08,  p  =  .01,  indicating  that  a  significant  improvement  in 
the  number  of  correct  commands  issued  and/or  actions  taken  during  the  incorrect  position  event 
had  occurred.  The  mean  score  on  this  variable  for  the  training  scenario  was  .81  commands 
issued  and/or  actions  taken,  and  for  the  testing  scenario  was  1.53  commands  issued  and/or 
actions  taken.  Figure  14  shows  a  graphic  representation  of  the  performance  of  trainees  on 
handling  of  the  incorrect  report  recognition  event  during  the  training  and  testing  scenarios. 
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Figure  14: 

Trainees’  Performance  on  Incorrect  Report  Recognition  During  the  VESUB  TEE 

No  significant  main  effect  was  foimd  for  the  between-subjects  factor  of  experience  level,  F(2, 
36)  =  2.17, 2  =  .129,  and  no  significance  was  foimd  for  the  interaction  of  scenario  session  and 
experience  level,  F(2, 36)  =  .429,  p  =  .654,  indicating  that  learning  was  not  affected  by  level  of 
experience  and  that  learning  occurred  across  all  levels  of  experience. 

Communications 


Three  variables  were  used  to  assess  the  trainees  learning  of  appropriate  communication  skills: 
commands  to  get  underway,  acknowledging  reports,  and  use  of  station  identifiers.  Commands  to 
get  underway  was  the  total  number  of  commands  correctly  issued  to  properly  get  own  ship 
underway  during  the  entire  scenario.  During  each  scenario  there  were  three  instances  where  the 
trainee  would  have  needed  to  issue  commands  to  get  own  ship  underway.  A  mixed  factorial 
ANOVA  did  not  result  in  a  significant  main  effect  for  scenario  session,  F(l,  36)  =  4.48,  p  =  .041, 
indicating  that  no  significant  improvement  in  the  number  of  commands  issued  to  get  own  ship 
underway  occurred  between  the  training  and  testing  scenarios.  A  significant  main  effect  was 
found  for  the  between-subjects  factor  of  experience  level,  F(2, 36)  =  5.43,  p  =  .009,  indicating 
that  prior  experience  in  ship  handling  did  have  an  affect  on  learning  effectiveness  in  this 
particular  task.  Figure  15  shows  a  graphic  representation  of  the  performance  of  trainees  on 
execution  of  proper  commands  issued  to  get  own  ship  underway  during  the  training  and  testing 
scenarios. 
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From  looking  at  the  graph  of  the  means  we  can  see  that  the  “less  than  1  year”  and  “1  to  3 
years”  experience  groups  both  performed  significantly  better  in  the  training  scenario  than  the 
“more  than  3  years”  group.  Although  the  “less  than  1  year”  and  the  “1  to  3  years”  groups  both 
improved  in  the  testing  scenarios,  this  improvement  was  not  significant. 
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Figure  15: 

Trainees’  Performance  on  Commands  to  Get  Underway  During  the  VESUB  TEE 

No  significance  was  foxmd  for  the  interaction  of  scenario  session  and  experience  level,  F(2, 
36)  =  .838,e  =  .441. 


Acknowledging  reports  was  a  ratio  of  the  number  of  times  reports  were  received  by  the 
trainee  and  were  correctly  acknowledged  compared  to  the  total  number  of  reports  received  by  the 
trainee.  A  mixed  factorial  ANOVA  did  not  result  in  a  significant  main  effect  for  scenario 
session,  F(l,  36)  =  .512,  p  =  .479,  indicating  that  no  significant  improvement  in  the  number  of 
reports  acknowledged  occurred  between  the  training  and  testing  scenarios  (training  scenario 
mean  =  .73  and  testing  scenario  mean  =  .76).  No  significant  main  effect  was  found  for  the 
between-subjects  factor  of  experience  level,  F(2, 36)  =  1.76,  p  =  .186.  No  significant  interaction 
between  scenario  session  and  experience  level  occurred  F(2,  36)  =  2.61,  p  =  .088. 

Use  of  station  identifiers  was  recorded  as  the  number  of  times  station  identifiers  were  not  used 
correctly.  A  mixed  factorial  ANOVA  resulted  in  a  significant  main  effect  for  scenario  session, 
F(l,  36)  =  42.42,  p  =  .000,  indicating  that  a  significant  improvement  in  the  use  of  station 
identifiers  did  occur  between  the  training  and  testing  scenarios.  The  mean  score  on  this  variable 
for  the  training  scenario  was  1.39  times  stations  identifiers  were  not  used  correctly,  and  for  the 
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testing  scenario  was  .57  times  station  identifiers  were  not  used  correctly.  Figure  16  shows  a 
graphic  representation  of  the  performance  of  trainees  on  issuance  of  proper  station  identifiers 
during  the  training  and  testing  scenarios. 
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Figure  16: 

Trainees’  Use  of  Station  Identifiers  During  the  VESUB  TEE 

No  significant  main  effect  was  found  for  the  between-subjects  factor  of  experience  level,  F(2, 
36)  =  1.29,  E  =  .286  and  no  significant  interaction  between  scenario  session  and  experience  level 
occurred  F(2,  36)  =  .309,  p  =  .736. 

Rules  of  the  Road 


One  variable  was  used  to  assess  the  trainees’  learning  and  comprehension  of  rules  of  the  road: 
ferry  passage.  Ferry  Passaee  was  the  total  number  of  commands  correctly  issued  to  properly  and 
safely  execute  a  port-to-port  passage  with  an  approaching  traffic  vessel  that  had  crossed  from 
right  to  left.  A  mixed  factorial  ANOVA  resulted  in  a  significant  main  effect  for  scenario  session, 
F(l,  36)  =  42.42,  p  =  .000,  indicating  that  a  significant  improvement  in  the  number  of  commands 
issued  to  safely  execute  a  port-to-port  passage  with  other  traffic  vessels  occurred  between  the 
training  and  testing  scenarios.  The  mean  score  on  this  variable  for  the  training  scenario  was  2.88 
commands  issued,  and  for  the  testing  scenario  was  4.43  commands  issued.  Figure  17  shows  a 
graphic  representation  of  the  performance  of  trainees  on  execution  of  proper  commands. 
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Figure  17: 

Trainees’  Performance  on  Rules  of  the  Road  Skills  During  the  VESUB  TEE 

No  significant  main  effect  was  found  for  the  between-subjects  factor  of  experience  level,  F(2, 
36)  =  .522, 2  =  .598  and  no  significant  interaction  between  scenario  session  and  experience  level 
occurred  F(2,  36)  =  .205,  p  =  .815. 

EXPERT  OPINION  DATA 

Opinions  on  the  VESUB  system  were  solicited  from  the  trainees  after  they  experienced  the 
orientation,  training,  and  testing  scenarios.  Additional  expert  opinions  were  collected  from 
eleven  persons  who  were  not  part  of  the  formal  TEE,  but  who  experienced  demonstration 
scenarios  and  observed  portions  of  the  TEE  (a  twelfth  observer  only  provided  open-ended 
comments). 

Trainees’  Opinions 

At  the  end  of  the  TEE,  each  trainee  was  given  an  opinion  questionnaire  to  fill  out  (Appendix 
G).  The  questionnaire  consisted  of  a  series  of  eight  forced-choice  questions  and  one  open-ended 
question.  Six  questions  referred  to  the  training  capability  and  functionality  of  the  VESUB 
system  and  two  questions  concerned  the  comfort  and  functionality  of  the  HMD.  The  forced- 
choice  questions  were  answered  by  circling  the  appropriate  answer  on  a  Likert  Scale  with  the 
choices  being  “strongly  agree,”  “agree,”  “neutral,”  “disagree,”  and  “strongly  disagree.” 
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1.  Positive  Learning  Experience.  The  majority  of  the  trainees  reacted  very  positively  to  the 
VESUB  system.  For  question  One,  “I  had  a  positive  learning  experience  in  the  VESUB  system,” 
28  out  of  42  trainees  (66.7%)  responded  that  they  “strongly  agreed,”  13  out  of  42  (31.0%) 
responded  that  they  “agreed”  and  1  out  of  42  (2.3%)  responded  that  they  were  “neutral.”  None 
of  the  trainees  responded  that  they  “disagreed”  or  “strongly  disagreed”  which  would  have 
indicated  that  they  did  not  have  a  positive  learning  experience  with  the  VESUB  system. 

2.  Increased  Confidence.  For  question  Two,  “Training  with  VESUB  will  increase  my 
confidence  in  my  ship  handling  skills,”  27  out  of  42  (64.3%)  responded  that  they  “strongly 
agreed,”  12  out  of  42  (28.6%)  responded  that  they  “agreed,”  and  2  out  of  42  (4.8%)  responded 
that  they  were  “neutral.”  Only  1  out  of  42  (2.3%)  responded  that  they  “disagreed”  and  none 
responded  that  they  “strongly  disagreed,”  indicating  that  over  97%  of  the  trainees  that 
participated  left  with  increased  confidence  in  their  ship  handling  skills. 

5.  Introductory  Training.  On  question  Five,  “VESUB  should  be  used  for  introductory  ship 
handling  training,”  34  out  of  42  (81.0%)  responded  that  they  “strongly  agreed,”  7  out  of  42 
(16.7%)  responded  that  they  “agreed,”  and  0  out  of  42  (0.0%)  responded  that  they  were 
“neutral.”  Only  1  out  of  42  (2.3%)  responded  that  they  “disagreed”  and  none  responded  that  they 
“strongly  disagreed,”  indicating  that  over  97%  of  the  trainees  that  participated  indicated  that  they 
felt  that  VESUB  would  be  an  excellent  tool  for  introductory  ship  handling  training. 

6.  Refresher  Training.  On  question  Six,  “VESUB  should  be  used  for  refresher  training  for 
ship  handling  skills,”  30  out  of  42  (71.4%)  responded  that  they  “strongly  agreed,”  9  out  of  42 
(21.4%)  responded  that  they  “agreed,”  and  3  out  of  42  (7.2%)  responded  that  they  were 
“neutral.”  None  responded  that  they  “disagreed”  or  that  they  “strongly  disagreed,”  indicating 
that  93%  of  the  trainees  that  participated  felt  that  VESUB  would  be  an  excellent  tool  for 
refresher  ship  handling  training. 

7.  Classroom  Only.  On  question  Seven,  “VESUB  should  only  be  used  in  the  classroom 
(shore  based)  training,”  5  out  of  42  (12.0%)  responded  that  they  “strongly  agreed,”  8  out  of  42 
(19.0%)  responded  that  they  “agreed,”  8  out  of  42  (19.0%)  responded  that  they  were  “neutral,” 

13  out  of  42  (31.0%)  responded  that  they  “disagreed,”  and  8  out  of  42  (19.0%)  responded  that 
they  “strongly  disagreed.”  These  results  show  that  the  trainees  had  very  mixed  opinions  about 
using  VESUB  exclusively  for  classroom  training. 

8.  Mission  Rehearsal.  Question  Eight  stated,  “VESUB  should  be  installed  on  submarines 
for  mission  rehearsal  and  refresher  training.”  Ten  out  of 42  (23.8%)  responded  that  they 
“strongly  agreed,”  13  out  of  42  (31.0%)  responded  that  they  “agreed,”  15  out  of  42  (35.7%) 
responded  that  they  were  “neutral,”  3  out  of  42  (7.1%)  responded  that  they  “disagreed,”  and  1  out 
of  42  (2.3%)  responded  that  they  “strongly  disagreed.”  These  results  indicate  that  although  most 
trainees  believe  VESUB  could  be  used  for  mission  rehearsal,  some  are  still  skeptical. 
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Figure  18  shows  a  graphical  representation  of  the  frequencies  of  responses  for  the  previous  six 
questions. 
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Figure  18: 

TEE  Trainees’  Opinions  on  the  VESUB  System 

The  remaining  two  questions  dealt  not  with  the  training  effectiveness  and  functionality  of  the 
system,  but  with  the  comfort  and  functionality  of  the  HMD  which  indirectly  may  have  affected 
the  training  effectiveness  and  functionality  of  the  VESUB  training  system  as  a  whole. 

3.  HMD  Visual  Adjustment.  On  question  Three,  “I  was  able  to  fully  adjust  the  VESUB 
HMD  to  meet  my  specific  visual  need,”  14  out  of 42  (33.3%)  responded  that  they  “strongly 
agreed,”  17  out  of 42  (40.5%)  responded  that  they  “agreed,”  6  out  of  42  (14.3%)  responded  that 
they  were  “neutral,”  and  5  out  of  42  (12.0%)  responded  that  they  “disagreed.”  None  responded 
that  they  “strongly  disagreed.” 

4.  HMD  Fit  Adjustment.  On  question  Four,  “I  was  able  to  fully  adjust  the  VESUB  HMD 
comfortably  to  fit  my  head,”  10  out  of 42  (23.8%)  responded  that  they  “strongly  agreed,”  17  out 
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of 42  (40.5%)  responded  that  they  “agreed,”  1 1  out  of  42  (26.2%)  responded  that  they  were 
“neutral,”  and  4  out  of  42  (9.5%)  responded  that  they  “disagreed.”  None  responded  that  they 
“strongly  disagreed.” 

Open-Ended  Question.  The  open-ended  question  allowed  the  trainees  to  provide  comments 
and  suggestions  on  any  topic  that  they  believed  would  improve  the  performance  or  training 
effectiveness  of  VESUB.  Below  is  a  listing  of  their  comments  grouped  into  content  areas.  The 
number  in  parentheses  following  some  comments  indicates  multiple  instances  of  similar 
comments. 

General  Reactions  to  System 

♦  I  learned  or  relearned  several  lessons  today. 

♦  Overall,  a  very  good  system.  Provides  realistic  training  in  an  environment  where  mistakes 
don’t  cost  a  crew  or  a  boat  (2). 

♦  A  very  good  product.  Great  tool.  Outstanding  system  (4). 

♦  Provides  a  strong  tool  for  improving  ship  handling  and  contact  management  for  OODs. 

♦  Training  is  very  effective  and  comfortable. 

♦  Excellent  training  experience  (2). 

♦  I  felt  off  balance.  We  have  a  seat  on  the  port  side  of  the  bridge. 

Instructional  Tools  or  Approaches 

♦  Setting  up  situations  where  the  Navigator  makes  incorrect  recommendations  is  a  good  idea 
(e.g.,  mark  turn  when  it  should  be  later  to  avoid  a  contact;  come  left  when  should  be  right). 

♦  It  helped  a  lot  to  be  able  to  practice  casualties  and  commands. 

Voice  and  Sounds 

♦  Unable  to  give  commands  while  system  is  speaking.  OOD  should  override  all  voices, 
waiting  for  system  to  finish  wastes  valuable  time.  Sometimes  unable  to  get  a  word  in 
edgewise.  Frustrating  (15). 

♦  Speech  recognition/synthesis  needs  to  be  faster  (3). 

♦  Expand  vocabulary  (e.g.,  “continue  right. . .,”) 

♦  Include  cardinal  headings  for  Helm. 

♦  Need  to  be  able  to  correct  spoken  errors  (e.g.,  “[incorrect  command],  correction,  [new 
command]”) 

♦  Include  virtual  Contact  Coordinator. 

♦  Eliminate  station  identifiers. 

♦  Reduce  redvmdant  repeat  backs. 

♦  Speech  recognition  is  excellent,  but  some  ships  may  have  different  standing  orders.  More 
flexibility  (3). 

♦  Need  to  be  able  to  give  multiple  commands  to  different  stations  simultaneously.  Need  to  re¬ 
key  the  mike  is  unrealistic  (3). 

♦  Improve  wind  noise  as  a  speed  cue. 

♦  Add  more  sotmds.  They  make  it  seem  more  realistic  (e.g.,  passing  boats  on  one  side  or  the 
other). 
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Visual  Scene  and  Visual  System 

♦  Include  a  switch  to  activate/deactivate  binoculars  rather  than  a  voice  command  to  speed  up 
and  reduce  confusion  (13). 

♦  Objects  at  a  distance  need  to  have  higher  resolution  to  reduce  need  for  binoculars. 

♦  Ambiguity  in  contact  visual  range  and  AOB  is  good  -  keep  it. 

♦  Visual  scene  was  a  bit  jittery  when  doing  a  visual  scan  -  increase  refresh  rate  (3). 

♦  Simulate  a  grease  pencil  to  check  off  courses  on  the  course  card  and  record  contacts  (2). 

♦  Chart  should  be  more  realistic  -  include  shoal  water  and  soundings. 

♦  Chart  not  marked  up  the  way  they  normally  are.  Too  much  information  and  difficult  to  find 
navigation  aids  (3). 

♦  Implement  virtual  body/feet  to  increase  realism  (2). 

♦  Place  Pilot,  CAPT,  etc.  in  cockpit  to  increase  realism. 

♦  Improve  bow  wave  as  a  speed  cue. 

♦  Adjust  wake  to  a  more  realistic  size. 

♦  Add  more  contacts. 

♦  Visual  contrast  is  poor  -  objects  which  are  readily  detectable  in  real  life  require  binoculars  in 
the  virtual  world. 

♦  No  real  sense  of  distance  and  poor  spacial  awareness. 

♦  Include  a  time  display  (for  3  minute  rule). 

Submarine  Model 

♦  Sub  seems  to  respond  too  fast  to  bell  orders. 

Use  of  the  System 

♦  Great  introductory  training  for  OODs. 

♦  Would  welcome  the  opportunity  to  train  JOs  all  the  way  up  to  COs  (from  SOBC  throughout 
career).  VESUB  would  be  effective  for  introductory  and  refresher  training  (6). 

♦  Recommend  that  all  ANAVS  and  members  of  the  Navigation  Team  train  on  VESUB  to 
enhance  their  skills  (2). 

♦  Combine  with  other  trainers  for  full  control  party  training. 

♦  Valuable  onboard  for  mission  rehearsal  in/out  of  unfamiliar  ports. 

♦  Great  to  teach  JOs  how  to  moor  a  ship  -  something  we  have  little  chance  to  do. 

♦  System  would  provide  extremely  good  training  with  an  experienced  training  team. 

♦  Setting  up  collision  scenarios  (e.g.,  Jacksonville)  and  giving  poor  recommendations  from  the 
Navigator  would  be  excellent  training. 

♦  You  could  replay  COMSUBLANT  Lessons  Learned  (collision/groimding  presentations)  to 
show  where  mistakes  were  made. 

♦  Make  VESUB  a  requirement  (similar  to  a  fast  cruise)  for  boats  that  have  been  in  port  for  a 
prescribed  number  of  days. 

♦  Must  reduce  size  of  system  if  it  is  to  be  brought  aboard  the  ship. 

♦  Could  be  useful  for  at-sea  training. 

♦  Sub-based  system  would  be  great  for  prepiloting  training  and  proficiency  training  during 
long  underway  time. 
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♦  Use  of  VESUB  to  practice  open-ocean  MOBs  could  be  useful  to  practice  recovery  by  own 
ship. 

♦  Add  additional  scenarios  (e.g.,  jammed  rudder,  open  ocean  with  other  contacts). 
Head-Mounted  Display 

♦  HMD  is  uncomfortable  after  a  while  -  consider  more  padding  on  the  top  piece  (2). 

♦  A  wider  field  of  view  would  help.  Had  to  move  head  more  than  in  real  life  (7). 

♦  Felt  fatigue  in  shoulders  and  upper  back  ifrom  holding  HMD. 

♦  Make  large  enough  to  fit  while  wearing  glasses. 

♦  Fit  pretty  well  but  shifted  when  I  looked  over  the  side. 

♦  Too  much  stray  light  in  at  the  bottom. 

♦  HMD  cord  sometimes  got  tangled  with  my  feet  -  perhaps  suspend  the  cord  off  of  the  deck. 

♦  HMD  might  benefit  jfrom  a  chin  strap. 

♦  The  rear  adjustment  popped  open  each  time  I  tried  to  tighten  it. 

♦  Problems  fitting  HMD  properly. 

♦  HMD  focus  was  in  a  position  that  I  bumped  it  each  time  I  tried  to  hold  HMD  steady. 

Haptic  Capabilities 

♦  Implement  virtual  hands  to  assist  OOD  (2). 

Observers*  Opinions 

Observers  of  the  TEE  filled  out  the  Observer  Questionnaire  (Appendix  H  ),  which  consisted 
of  six  forced-choice  questions  and  one  open-ended  question.  The  forced-choice  questions  were  a 
variation  of  the  questions  answered  by  the  trainees,  but  without  reference  to  HMD  fit  and 
adjustments.  Figure  19  is  a  graphical  representation  of  the  responses  to  the  observer  questions. 

1.  Positive  Learning.  Question  One  stated,  “The  VESUB  system  provides  a  positive 
learning  experience.”  As  can  be  seen  in  Figure  19,  all  of  the  observers  either  “agreed”  (3  of  1 1; 
27%)  or  “strongly  agreed”  with  this  statement  (8  of  1 1 ;  73%). 

2.  Increased  Confidence.  On  question  Two,  “Training  with  VESUB  will  increase  a 
trainee’s  confidence  in  his  ship  handling  skills,”  5  out  of  1 1  (45.5%)  “strongly  agreed,”  5 
(45.5%)  “agreed,”  and  1  (9%)  was  “neutral.” 

3.  Introductory  Training.  On  question  Three,  “VESUB  should  be  used  for  introductory 
ship  handling  training,”  6  out  of  1 1  (54.5%)  “strongly  agreed,”  3  (27%)  “agreed,”  and  2  (18%) 
were  “neutral.” 

4.  Refresher  Training.  Question  four  stated,  “VESUB  should  be  used  for  refresher  training  for 
ship  handling  skills.”  With  the  exception  of  one  neutral  observer  (9%),  everyone  either  strongly 
agreed  (54.5%)  or  agreed  (4  of  1 1;  36%). 
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Figure  19: 

TEE  Observers’  Opinions  on  the  VESUB  System 


5.  Classroom  Only.  On  question  Five,  “VESUB  should  only  be  used  in  classroom  (shore- 
based  training),”  1  of  1 1  (9%)  “strongly  agreed,”  1  (9%)  “agreed,”  3  (27%)  were  “neutral,”  4 
(36%)  “disagreed,”  and  2  (18%)  “strongly  disagreed.” 

6.  Mission  Rehearsal.  The  sixth  and  final  forced-choice  question  stated,  “VESUB  should  be 
installed  on  submarines  for  mission  rehearsal.”  Most  observers  either  “agreed”  (5;  45.5%)  or 
were  “neutral”  (4;  36%)  on  this  question.  One  person  (9%)  “strongly  agreed”  and  1  (9%) 
“disagreed.” 

Open-Ended  Question.  The  open-ended  question  allowed  the  observers  to  discuss  any  topic 
they  believed  would  improve  VESUB.  Below  is  a  listing  of  their  comments,  broken  into  content 
areas. 

General  Reactions  to  System 

♦  This  trainer  has  excellent  potential. 

♦  This  is  an  excellent  system. 
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System  Expansions 

♦  Provide  an  open  ocean  scenario  database. 

♦  Mooring  and  docking  would  be  a  GREAT  benefit. 

♦  Integration  into  various  teams  (maneuvering  watch,  piloting)  is  an  area  with  value. 

♦  Use  for  maintenance  training  (e.g.,  torpedo  handling). 

Instructional  Tools 

♦  Use  red  to  highlight  red  buoy  edge  of  chaimel,  green  for  green  buoy  edge. 

Voice  and  Sounds 

♦  Improve  background  soimds,  both  fi-om  environment  and  ship. 

♦  Verbal  interface  needs  work.  Student  can’t  be  heard  while  the  system  is  speaking. 

♦  Include  more  non-universal  orders. 

♦  Faster  order  input  or  more  variable. 

♦  Drop  “helm-bridge”  fi'om  voice  commands. 

♦  Voice  of  Nav/Helm  is  choppy,  separate  words  fi'om  numbers. 

♦  Navigator  courses  are  difficult  to  imderstand.  Pause  before  the  course. 

♦  Binocular  command  should  be  manual. 

Visual  Scene  and  Visual  System 

♦  Include  area  for  OOD  to  write  down  data  (grease  pencil  on  windshield). 

♦  Field  of  view  in  HMD  needs  to  be  wider. 

♦  A  bridge-to-bridge  radio  is  needed. 

♦  Radar  Repeater,  GPS,  Etc.  would  be  effective. 

♦  Contact  card  for  bridge-to-bridge  communications. 

♦  Visual  clarity  of  objects  diminishes  with  range  faster  than  real  life. 

♦  Improve  refi-esh  rate  of  visuals  when  trainee’s  head  moves. 

Virtual  Crew 

♦  Include  Contact  Coordinator  in  virtual  crew. 

Addition  of  Haptic  Capabilities 

♦  Make  the  alidade  more  interactive. 

♦  Bring  the  subject’s  hands  into  play. 

♦  Include  use  of  a  single  VR  glove. 
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DISCUSSION 


SIMULATION  SIDE  EFFECTS 

The  use  of  virtual  environments  for  future  training  systems  will  depend  not  only  on  the 
implementation  for  specific  tasks,  but  also  on  whether  trainees  can  learn  in  the  virtual 
environment.  Learning  can  not  occur  if  the  trainee’s  interface  with  the  virtual  world  causes  side 
effects  that  distract  the  trainee.  The  VESUB  TEE  was  one  of  the  most  extensive  evaluations  of  a 
virtual  environment  training  system  yet  to  be  conducted.  Each  trainee  experienced  three 
scenarios  in  the  virtual  environment  for  a  total  of  almost  2  hours.  It  was  also  one  of  the  first  to 
be  conducted  in  a  training  facility  with  Navy  trainees  and  operational  personnel  rather  than  a 
laboratory  with  college  students.  The  extent  of  simulation  side  effects  during  this  TEE  could 
influence  the  use  of  VE  for  other  applications. 

Of  the  41  subjects  who  participated  in  the  TEE,  two  experienced  symptoms  that  were  severe 
enough  that  they  requested  to  be  removed  from  the  system.  Data  collectors,  monitoring  the 
subjects’  physical  conditions,  had  prompted  the  subjects  to  request  removal  fi-om  the  system. 

Both  subjects  were  from  the  Norfolk  subject  pool  (N-7  and  N-14).  Subject  N-7  was  removed 
after  10  minutes  and  10  seconds  of  the  training  scenario.  Subject  N-14  lasted  for  twenty-three 
minutes  and  fifteen  seconds  of  the  training  scenario.  Subject  N-7  reported  moderate  levels  of 
general  discomfort,  sweating,  and  stomach  awareness,  and  a  severe  level  of  increased  salivation. 
Subject  N-14  reported  moderate  levels  of  headache,  and  slight  levels  of  general  discomfort,  eye 
strain,  difficulty  focusing,  difficulty  concentrating,  stomach  awareness,  and  burping.  Both 
subjects  reported  moderate  levels  of  Nausea. 

One  additional  subject  asked  to  be  removed  from  the  system  after  only  about  two  minutes  of 
the  orientation  scenario  and  was  immediately  replaced.  An  interview  with  this  subject  revealed 
that  he  is  very  sensitive  to  viewing  any  computer  screen.  He  reported  that  he  becomes  diz2y 
when  watching  his  son  play  video  games.  He  was,  therefore,  considered  an  anomaly  and  his 
scores  were  not  included  in  the  side  effects  data. 

Often,  researchers  using  the  Simulator  Sickness  Questionnaire  (SSQ)  use  the  preexposure 
data  as  a  screening  mechanism  to  eliminate  persons  who  arrive  at  the  simulator  with  some 
preexisting  condition  (Kennedy,  et  al.,  1993).  For  the  purposes  of  this  report,  the  preexposure 
data  are  included  and  no  subjects  were  eliminated  from  further  analyses.  Persons  who  wish  to 
take  a  more  conservative  approach  may  use  the  raw  data  found  in  Appendix  K. 

*  The  frequency  of  the  Total  SSQ  scores  are  shown  in  Table  2  and  Figure  2.  Over  half  of  the 

participants  showed  zero  or  low  (1 1.22  and  below)  Total  SSQ  scores  on  the  pretest  and  after  each 
scenario.  Most  other  scores  fall  in  the  moderate  range  (14.96  -  37.4).  Although  the  frequency  of 
higher  Total  SSQ  scores  increases  after  the  scenarios,  only  two  cases  of  very  high  scores  were 
reported.  These  were  during  the  training  scenario  from  the  two  individuals  who  were  removed 
from  the  system.  Most  individuals  had  no  trouble  staying  in  the  system  for  the  entire  TEE  and 
experienced  only  low  to  moderate  side  effects. 
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The  mean  subfactor  scores  are  shown  in  Figure  3.  All  subfactors  increase  after  the  scenarios. 
The  Nausea  subfactor  is  relatively  low  after  the  orientation,  but  increases  after  the  training 
scenario.  This  is  because  the  data  from  the  two  subjects  who  were  removed  during  the  training 
scenario  inflate  these  scores.  After  the  testing  scenario  (without  these  two  subjects),  the  Nausea 
subfactor  score  falls  to  slightly  below  the  level  after  the  orientation  scenario.  It  appears  that, 
with  the  exception  of  highly  sensitive  persons,  nausea  does  not  present  a  problem  in  VESUB. 

The  Oculomotor  subfactor  scores  follow  a  similar  pattern.  Although  they  do  not  drop  as  low 
as  the  Nausea  scores,  the  mean  oculomotor  score  after  the  testing  scenario  is  lower  than  after  the 
training  scenario,  having  dropped  almost  to  the  same  level  as  after  the  orientation  scenario. 
Although  not  something  to  ignore,  it  is  likely  that  improvements  in  HMD  design  will  lead  to 
fewer  oculomotor  problems  in  VE  systems. 

The  Disorientation  subfactor  scores  are  also  higher  after  the  training  scenario,  but  fall  to  a 
lower  level  after  the  testing  scenario  than  their  level  after  the  orientation  scenario.  This  may  be 
due  to  subjects  becoming  more  familiar  with  the  narrow  field  of  view  in  the  HMD.  By  learning 
to  use  more  head  movements  and  also  becoming  more  familiar  with  the  use  of  navigation  aids, 
subjects  probably  experienced  less  disorientation.  Nevertheless,  it  is  expected  that  HMDs  with 
wider  fields  of  view  (both  horizontally  and  vertically)  will  lead  to  improvements  in  this  area. 
Reductions  in  the  number  and  size  of  head  movements  could  also  lead  to  improvements  in  the 
oculomotor  and  nausea  areas. 

Even  though  individuals  were  exposed  to  a  VE  for  a  total  time  of  almost  two  hours,  the 
incidence  of  simulation  side  effects  diuing  the  VESUB  TEE  was  low  to  moderate.  From  these 
data,  there  is  no  reason  to  assume  that  VEs  will  produce  simulation  side  effects  at  a  greater  rate 
than  any  other  form  of  simulation.  Furthermore,  with  longer  exposure,  individuals  appear  to 
acclimate  to  the  system.  Improvements  in  hardware  and  software  should  reduce  these  side 
effects  even  more.  One  possible  side  effects  problem  may  occur  if  VE  systems  are  deployed  on 
ships  at  sea  due  to  asynchrony  between  ship  motion  and  apparent  motion  in  the  VE.  However, 
since  most  deployed  training  will  be  done  while  the  ship  is  in  port  (i.e.,  moored  at  a  pier),  VE 
systems  should  still  be  a  viable  training  alternative. 

SHIP  HANDLING  PERFORMANCE 

Ship  handling  is  a  very  complex  task,  requiring  the  development  of  a  comprehensive  mental 
model  of  the  task  situation  and  the  application  of  numerous  sMlls  supported  by  various 
combinations  of  the  perceptual  and  cognitive  components  of  “Seaman’s  Eye”  (see  Table  1).  The 
improvement  of  trainees’  ship  handling  performance  during  the  TEE  was  measured  on  fifteen 
variables  (see  Table  3),  grouped  into  seven  performance  categories.  Each  of  the  performance 
variables  used  in  the  TEE  required  unique  combinations  of  the  perceptual  and  cognitive  skills. 
The  most  relevant  supporting  components  for  successful  performance  on  each  variable  are  used 
to  focus  the  discussions  that  follow,  along  with  less  detailed  discussions  of  other  supporting 
components. 

Measurement  on  some  variables  were  simple  numbers  (e.g.,  time  to  issue  a  command  or  the 
difference  between  reported  and  actual  positions).  Other  variables  were  measured  using  ratios 
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(e.g.,  number  of  correct  actions  taken  to  number  of  possible  correct  actions).  Ratios  were  used  to 
account  for  minor  differences  between  inboxmd  and  outbound  scenarios  (e.g.,  number  of  contacts 
available,  number  of  turns  required,  etc.).  Finally,  performance  on  some  variables  were  rated  as 
satisfactory  or  unsatisfactory  when  more  than  one  “correct”  answer  was  possible  (e.g.,  reported 
position  could  be  distance  from  the  next  turn,  distance  from  a  navigation  aid,  distance  from  a 
buoy,  etc.). 

Position  Determination 


Determination  of  the  position  of  own  ship  is  a  task  that  must  be  accomplished  each  time  the 
OOD  assumes  the  watch.  Further  actions  during  the  watch  will  all  depend  on  the  accuracy  of  the 
OOD’s  estimation  of  the  cmrent  and  anticipated  location  of  own  ship.  There  are  a  variety  of 
navigation  aids  and  geographic  features  that  can  be  used  by  the  OOD  to  determine  position.  The 
specific  navigation  aids  and  geographic  features  used  were  recorded  by  the  research  team  as  each 
trainee  made  his  determination  of  position.  Then,  guidance  to  improve  the  use  of  the  navigation 
aids  was  provided  during  the  instructional  intervention  segments  of  the  training  scenario.  Two 
variables  were  used  to  assess  improvements  in  trainees’  performance  in  this  skill  area:  position 
of  own  ship  across  the  track  and  position  of  own  ship  along  the  track. 

Position  of  Own  Ship  Across  Track.  This  variable  measured  the  trainee’s  skills  in 
determining  the  ship’s  position  left  or  right  of  the  center  of  the  channel.  This  required  the  trainee 
to  visually  sight  the  range  markers  either  forward  or  astern  and  use  them  in  conjunction  with 
other  navigation  aids  and  charts  to  make  this  judgment.  The  measurement  was  the  difference 
between  the  trainee’s  reported  distance  and  actual  distance  from  track. 

When  using  range  markers  to  determine  own  ship  location  across  the  track,  orientation  can  be 
tricky.  Figure  20  helps  clarify  this  procedure.  When  viewing  range  markers  ahead,  if  the  front 
marker  is  to  the  left  of  the  rear  marker,  then  own  ship  is  right  of  range  and  right  of  track.  When 
viewing  range  markers  astern,  if  the  front  marker  is  to  the  right  of  the  rear  marker,  then  own  ship 
would  be  to  the  left  of  range,  but  to  the  right  of  track.  The  (hstance  left  or  right  of  track  can  be 
determined  relatively  easily.  Knowing  the  width  of  the  channel  (e.g.,  from  the  scale  on  the  chart) 
one  knows  the  maximum  distance  own  ship  could  be  from  the  centerline.  Combining  this 
knowledge  with  distances  from  buoys  and  navigation  aids  nearby,  allows  the  OOD  to  fine  tune 
his  estimate  of  distance  from  the  center  of  the  channel.  Monitoring  of  own  ship’s  wake  can  also 
give  the  OOD  a  general  indication  of  his  location  in  relation  to  the  centerline  and  navigation  aids. 

Two  perceptual  components  and  one  cognitive  component  of  “Seaman’s  Eye”  are  most 
relevant  to  this  task.  Components  IP  (Locating  and  Identifying  Navigation  Aids)  and  2P 
(Judging  Distance)  require  the  trainee  to  survey  the  visual  scene  to  find  and  evaluate  the 
available  navigation  aids.  The  OOD  must  determine  which  of  a  variety  of  range  markers  are  the 
markers  for  the  leg  of  the  channel  he  is  ctirrently  on  before  he  can  be  sure  that  he  is  making  the 
correct  interpretation  from  their  alignment.  To  further  specify  location,  the  OOD  must  use 
component 
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Range  Markers  Ahead; 
Shows  Own  Ship  is 
right  of  range  & 
right  of  track  (forward 
range  is  to  the  left  of 
the  rear  range) 


Rear 

Ranges 


Range  Markers  Astern; 
Shows  Own  Ship  is 
left  of  range  & 
right  of  track  (forward 
range  is  to  the  right  of 
the  rear  range) 


Figure  20: 

Use  of  Range  Markers  to  Determine  Own  Ship’s  Position  Across  Track 


1C  (Understanding  the  Relationship  of  Visual  Cues  to  their  Representation  on  Charts).  By 
consulting  the  scale  on  the  chart,  the  trainee  can  determine  the  width  of  the  channel  and  by 
locating  specific  navigation  aids  (e.g.,  finding  the  numbers  on  a  day  marker),  he  can  fine  tune  his 
judgment  of  location  across  the  channel.  Less  critical  than  the  other  components,  but  still 
relevant  to  the  task  is  component  2C  (Understanding  Relative  Size  and  Height/Range 
Relationships).  This  cognitive  component  helps  the  OOD  determine  own  ship’s  distance  firom 
navigation  aids  based  on  his  perception  of  their  size  in  the  visual  scene  and  his  ability  to  relate 
these  cues  to  distance  estimation. 

Significant  learning  was  demonstrated  on  this  variable  for  subjects  at  all  experience  levels. 
This  indicates  that  VE  technologies  can  be  effectively  used  for  both  introductory  and  refiresher 
training  for  this  aspect  of  position  determination.  Trainees  were  able  to  improve  their  location 
and  use  of  navigation  aids  and  other  cues  to  determine  their  location  relative  to  the  center  of  the 
channel.  This  skill  is  especially  important  in  harbors  like  Kings  Bay,  because  it  is  very  narrow 
with  shallow  water  on  either  side  of  the  channel.  The  submarine  OOD  does  not  have  much  room 
for  error  when  own  ship  is  moving  rapidly  through  such  an  imforgiving  channel,  making  the 
timely  and  correct  use  of  navigation  aids  even  more  essential  for  the  safe  handling  of  the  ship. 

Position  of  Own  Ship  Along  Track .  In  addition  to  determining  whether  the  ship  is  left  or 
right  of  track,  the  OOD  must  also  locate  the  ship’s  position  along  the  track.  This  variable  was  a 
rating  of  the  accuracy  of  the  trainee’s  judgment  of  this  position.  A  satisfactory  rating  was  given 
if  the  trainee’s  reported  position  was  within  200  yards  (the  approximate  length  of  a  Trident 
submarine)  of  the  actual  location  of  the  ship.  Otherwise,  the  trainee  received  an  unsatisfactory 
rating. 
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When  determining  the  location  of  own  ship,  utilizing  all  available  sources  of  information  can 
not  be  overemphasized.  Various  combinations  of  geographic  features  and  navigation  aids  can  be 
used  in  this  task,  including:  relative  bearings  from  own  ship  to  objects  and  geographic  features; 
proximity  to  navigational  aids  and  buoys;  own  ship’s  size;  channel  width;  distance  to  land;  and 
chart  scale.  For  example,  if  own  ship  was  abeam  (next  to)  a  particular  navigation  aid,  its  number 
could  be  observed  and  foimd  on  the  chart.  Next,  the  relative  bearing  to  a  second  navigation  aid 
or  geographic  feature  could  be  used  to  more  accurately  define  the  ship’s  specific  location. 

Adding  another  buoy  in  the  vicinity  would  further  improve  the  accuracy. 

Judging  distance  is  another  significant  factor  in  determining  the  position  of  own  ship.  The 
size  of  own  ship  (i.e.,  a  Trident  is  564  feet  long)  can  be  used  as  a  visual  cue  to  help  judge 
distances  to  objects.  For  example,  some  trainees  used  the  Trident  that  was  located  in  the 
magnetic  silencing  facility  (MSF)  to  help  estimate  how  far  their  own  ship  was  from  the  MSF. 
Familiarization  with  harbors  can  also  aid  in  determining  one’s  position  by  making  it  easier  to 
find  various  navigation  aids  and  relate  them  to  each  other  and  to  own  ship’s  location. 

The  same  perceptual  and  cognitive  components  used  to  judge  position  across  track  are  also 
used  to  judge  the  position  of  own  ship  along  the  track.  More  emphasis  is  placed  on  the  cognitive 
component  1C  (Understanding  the  Relationship  of  Visual  Cues  to  their  Representations  on 
Charts),  because  the  OOD  must  use  the  chart  more  extensively  to  locate  specific  navigation  aids 
and  determine  own  ship’s  distance  from  them. 

No  significant  improvement  in  performance  was  found  on  this  variable.  This  is  probably 
because  the  measure  (satisfactory  if  within  200  yards  of  actual  position)  was  not  sensitive  enough 
to  detect  small  improvements  in  this  skill  area.  Most  of  the  trainees  were  able  to  use  enough 
cues  from  the  virtual  scene  to  obtain  a  satisfactory  rating  during  both  the  training  and  testing 
scenarios.  Nevertheless,  the  VESUB  system  did  demonstrate  that  the  critical  visual  cues  can  be 
presented  with  enough  fidelity  to  support  this  task.  More  sensitive  performance  measurement 
criteria  for  this  variable  should  be  used  in  the  future.  For  example,  the  trainee  could  be  required 
to  identify  own  ship’s  position  by  reference  to  a  specific  location  on  the  chart.  That  location 
could  then  be  compared  to  own  ship’s  actual  position  to  obtain  a  precise  niuneric  distance 
between  actual  and  reported  positions. 

Contact  Management 

Contact  management  includes  the  ability  to  identify,  prioritize,  and  avoid  contacts.  It  is  a 
critical  skill  area  that  the  OOD  must  master  to  safely  bring  the  ship  into  and  out  of  port.  The 
OOD  must  develop  and  maintain  a  mental  model  of  the  dynamic  relationship  of  all  contacts  to 
own  ship.  More  complex  tasks,  such  as  determining  a  contact’s  closest  point  of  approach  (CPA) 
and  its  relative  motion,  all  depend  on  the  accuracy  and  detail  of  the  OOD’s  mental  model.  Two 
variables  were  used  to  measure  performance  in  this  skill  area:  contacts  found  and  contacts  “of 
concern.” 

Contacts  Found  .  This  variable  was  the  ratio  of  the  total  number  of  contacts  the  trainee  was 
able  to  find  in  the  visual  scene  to  the  number  of  contacts  that  could  possibly  be  found.  Contacts 
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were  placed  at  various  ranges  and  locations  throughout  the  full  360  degree  visual  scene.  This 
forced  the  trainee  to  visually  search  for  the  contacts  and  use  binoculars  as  he  would  in  the  real 
world. 

Perceptual  component  4P  (Locating,  Identifying,  and  Avoiding  Obstacles)  is  most  relevant  in 
finding  contacts.  Another  perceptual  component  (2P:  Judging  Distance)  and  one  cognitive 
component  (2C:  Understanding  Relative  Size  and  Height/Range  Relationships,  and  Angle  on.  the 
Bow)  are  relevant  to  a  lesser  degree. 

No  main  effect  for  scenario  session  or  experience  level  was  found  on  this  variable.  However, 
there  was  a  significant  interaction  between  scenario  session  and  experience  level.  Inexperienced 
trainees  (less  than  one  year)  showed  the  most  improvement  in  performance.  Trainees  with 
moderate  experience  (1  to  3  years)  showed  approximately  the  same  performance  in  both  the 
training  and  testing  scenarios.  The  most  experienced  trainees  (over  3  years)  actually  showed  a 
decline  in  performance.  This  may  be  because  the  most  experienced  trainees  did  not  take  the 
simulated  activities  as  seriously  as  the  inexperienced  trainees.  They  may  not  have  taken  the  time 
to  fully  explore  the  virtual  world  to  locate  contacts  as  they  would  have  in  the  real  world. 
Experienced  instructors  often  observe  “traineritus”  in  some  trainees.  This  is  the  bias  that  some 
individuals  bring  to  training  sessions.  They  know  it’s  not  the  real  world,  so  they  may  not  try  as 
hard  and  take  the  scenarios  as  seriously  as  they  should.  Some  even  try  to  role  play  or  talk  their 
way  through  the  scenario  event  rather  than  taking  the  correct  actions.  A  number  of  experienced 
persons,  after  making  errors  and  receiving  instruction,  commented,  “I  knew  that,  but  didn’t  think 
it  was  important  in  the  simulation.” 

The  research  team  observed  increased  errors  in  the  size  and  distance  estimations  for  contacts 
at  larger  distances.  The  expectations  of  experienced  trainees  on  contact  appearance  might  have 
biased  them  in  locating  distant  contacts.  Several  contacts  were  purposely  placed  at  large 
distances  or  behind  partial  obstructions.  The  most  experienced  trainees  may  have  had  the 
expectation  that  the  simulated  contacts  would  be  easier  to  see  and,  therefore,  overlooked  some 
that  were  found  by  the  inexperienced  trainees,  who  did  not  have  these  expectations  and  searched 
more  diligently.  This  may  also  be  an  indication  that  the  fidelity  of  the  contacts  needs  to  be 
improved  to  make  them  as  easy  to  find  as  in  the  real  world. 

Contacts  “Of  Concern”.  In  addition  to  locating  contacts,  the  OOD  must  make  a 
determination  if  a  contact  is  “of  concern”  in  relation  to  own  ship’s  intended  track.  A  contact  can 
be  “of  concern”  because  of  its  location  (e.g.,  in  the  channel  in  front  of  own  ship),  its  direction  of 
movement  (e.g.,  moving  toward  own  ship),  its  ship  type  (e.g.,  sail  boats  can’t  maneuver  quickly 
so  they  must  be  closely  monitored),  or  a  variety  of  other  reasons.  This  variable  was  the  ratio  of 
the  total  number  of  contacts  the  trainee  identified  as  “of  concern”  to  the  number  of  contacts  the 
research  team  identified  as  “of  concern.” 

Determining  if  a  contact  is  “of  concern”  involves  several  perceptual  and  cognitive 
components  of  “Seaman’s  Eye.”  The  most  relevant  components  are  4P  (Locating,  Identifying, 
and  Avoiding  Obstacles)  and  7C  (Understanding  Methods  to  Differentiate  and  Prioritize  Traffic 
Contacts).  Less  relevant  components  are  2P  (Judging  Distance),  2C  (Understanding  Relative 
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Size/Height  Relationships  and  Angle  on  the  Bow),  5C  (Understanding  Rules  of  the  Road),  and 
6C  (Understanding  Relative  Motion). 

Significant  learning  was  demonstrated  on  this  variable  for  all  experience  levels.  This  would 
be  expected  with  inexperienced  trainees,  but  is  somewhat  surprising  for  the  more  experienced 
participants.  A  crucial  constituent  of  this  improvement  for  all  trainees  was  the  recognition  that 
even  an  “assist  tug,”  working  closely  with  own  ship,  should  be  considered  “of  concern.”  Many 
of  the  trainees,  at  all  experience  levels,  overlooked  this  with  comments  like  “he’s  working  with 
me”  or  “he  knows  what  I’ll  be  doing.”  During  the  instructional  intervention,  it  was  explained  to 
the  trainees  that  the  tug  is  still  “of  concern”  because  its  position  in  close  proximity  to  own  ship 
places  limits  on  own  ship  maneuverability  and  the  OOD’s  reaction  time  in  emergency  situations. 
The  key  lesson  is  that  the  OOD  should  never  become  complacent  when  dealing  with  any  contact. 

Another  area  of  improvement  was  the  recognition  that  small  pleasure  boats  are  almost  always 
“of  concern.”  During  the  training  scenario,  comments  like  “he’ll  get  out  of  the  way”  or  “I’m  the 
deep  draft  vessel  with  the  right  of  way”  were  given  as  explanations  for  lack  “of  concern”  about 
these  craft.  As  part  of  the  instructional  intervention,  it  was  pointed  out  that  one  can  never  make 
assumptions  about  what  the  other  craft  will  do  or  whether  the  persons  on  that  craft  understand 
right  of  way.  In  contact  interaction,  all  vessels  are  responsible  to  take  early  and  substantial 
actions  to  avoid  collisions.  Understanding  this  information  contributed  to  improvements  during 
the  testing  scenario,  probably  due  to  the  trainees’  more  comprehensive  understanding  of  the 
concept  “of  concern.” 

General  Ship  Handling 

A  variety  of  general  ship  handling  skills  are  required  to  maneuver  the  ship  through  the  harbor 
channel.  Three  variables  were  used  to  measure  performance  in  this  skill  area:  turning 
commands,  visually  checking  the  rudder,  and  visually  checking  ranges. 

Turning  Commands.  The  OOD  must  give  the  command  to  turn  at  the  correct  time,  in  the 
correct  direction,  and  with  the  correct  amount  of  rudder.  These  are  all  determined  by  prevailing 
conditions,  such  as:  ship’s  speed,  ship’s  maneuvering  characteristics  (advance  and  transfer),  set 
and  drift,  tides  and  currents,  position  left  or  right  of  track,  other  traffic,  etc.  The  navigator  will 
normally  prompt  this  turn  command  with  a  report,  “Mark  the  turn,”  based  on  the  navigation 
team’s  assessment  of  these  conditions.  The  OOD  must  determine  if  this  “mark”  is  correct  and 
quickly  take  action.  This  variable  was  the  ratio  of  the  total  number  of  correct  commands  given 
by  the  trainee  on  all  turns  throughout  the  scenario  to  the  number  of  possible  correct  commands. 

This  variable  involves  a  large  number  of  perceptual  and  cognitive  components  since  so  many 
elements  interact  in  the  decision  process  used  by  the  OOD  in  making  a  turn.  The  most  relevant 
perceptual  components  are  3P  (Identifying  the  Start  and  Completion  of  Turns)  and  5P  (Sense  of 
Ship’s  Responsiveness).  The  most  relevant  cognitive  components  are  3C  (Understanding 
Advance  and  Transfer)  and  12C  (Understanding  Effective  Communication  Procedures).  Two 
additional  perceptual  and  three  cognitive  components  are  also  relevant  to  this  variable:  IP 
(Locating  and  Identifying  Navigation  Aids),  2P  (Judging  Distance),  1C  (Understanding  the 
Relationship  of  Visual  Cues  to  their  Representations  on  Charts),  4C  (Understanding  the  Effects 
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of  Tides,  Currents,  Wind,  and  Seas),  and  IOC  (Understanding  Correct  Operation  of  Ship’s 
Systems). 

Significant  improvement  in  the  use  of  turning  commands  was  demonstrated  for  all  experience 
levels.  All  trainees  executed  more  correct  commands  for  turns  during  the  testing  scenario  than 
during  the  training  scenario.  One  reason  for  this  improvement  is  that  the  feedback  after  the 
training  scenario  stressed  that  the  OOD  should  plan  ahead  and  be  ready  for  the  turn.  Even  if  the 
OOD  has  forgotten  the  next  course,  when  the  Navigator  says,  “Mark  the  turn,”  the  OOD  should 
immediately  give  the  rudder  order  (e.g.,  “Right  15  degrees  rudder.”),  then  consult  his  chart  or 
course  card  and  give  the  course  command  (e.g.,  “Steady  course  268.”).  If  the  OOD  waits  to  give 
the  rudder  command,  precious  time  has  been  wasted  and  the  ship  could  be  placed  in  grave  danger 
before  the  OOD  realizes  it.  During  the  feedback  session,  each  trainee  was  reminded  of  the 
“Three  Minute  Rule.”  This  rule  of  thumb  is  used  to  help  the  OOD  understand  how  far  he  is 
moving  at  different  speeds.  It  states: 

The  distance  a  ship  travels  in  three  minutes  (in  yards)  is  its  speed  (in  knots)  multiplied  by  100. 

For  example,  if  own  ship  is  traveling  at  12  knots,  it  travels  1200  yards  in  three  minutes,  400 
yards  in  one  minute,  or  100  yards  every  15  seconds.  Thus,  if  the  trainee  delays  issuing  a  turning 
command  for  15  seconds  and  the  ship  is  traveling  at  12  knots,  the  ship  will  be  100  yards  beyond 
the  point  where  the  turn  should  have  begun.  In  a  narrow  channel,  such  as  Kings  Bay,  this  delay 
could  place  the  ship  in  extremis.  This  type  of  instructional  feedback,  combined  with  the 
experiences  gained  during  the  training  scenario,  prepared  the  trainees  for  better  performance 
dming  the  testing  scenario. 

Visually  Checking  the  Rudder.  Correct  protocol  requires  the  OOD  to  visually  check  the 
rudder  each  time  a  rudder  command  is  issued.  This  is  to  ensure  that  the  rudder  is  moving  in  the 
commanded  direction  so  corrections  may  be  made  before  the  ship  moves  into  danger.  This 
variable  was  the  ratio  of  the  total  number  of  times  the  trainee  visually  checked  the  rudder  to  the 
total  number  of  rudder  commands  given.  The  cognitive  component  most  relevant  to  this  variable 
is  IOC  (Understanding  Correct  Operation  of  Ship’s  Systems). 

Improvement  on  this  variable  was  demonstrated  for  all  experience  levels.  This  indicates  that 
even  experienced  individuals  can  forget  this  important  task,  yet  their  skills  can  be  quickly 
refireshed.  Initially,  many  trainees,  experienced  and  inexperienced,  attempted  to  solely  use  the 
rudder  indicator  on  the  simulated  bridge  suitcase  rather  than  turn  around  and  visually  find  the 
rudder.  They  were  reminded  that  this  indicator  can  malfimction  and  that,  as  the  OOD,  they 
should  rely  on  their  own  eyes  for  this  and  many  other  tasks.  The  VESUB  system  allowed  the 
trainee  to  turn  to  the  stem,  move  aroimd  visual  obstacles,  and  find  the  rudder.  It  is  likely  that  this 
“full-body”  experience  reinforced  the  verbal  instruction  to  help  “automate”  this  critical  activity. 

Visually  Checking  Ranges.  After  completing  a  turn  and  coming  to  a  new  course,  the  OOD 
must  visually  check  the  ship’s  position  right  or  left  of  the  center  of  the  channel  by  using  the 
range  markers  foimd  either  forward  or  astern  on  each  leg  of  the  harbor  channel.  This  variable 
was  the  ratio  of  the  total  number  of  times  the  ranges  were  visually  checked  to  the  total  number  of 
times  they  should  have  been  checked.  Perceptual  component  IP  (Locating  and  Identifying 
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Navigation  Aids)  is  the  most  relevant  to  this  variable,  with  cognitive  component  IOC 
(Understanding  Correct  Operation  of  Ship’s  Systems)  contributing  to  a  lesser  degree. 

The  significant  main  effect  of  training  session  demonstrated  that  learning  occurred  on  this 
variable  for  all  trainees.  There  was  also  a  significant  interaction  between  scenario  session  and 
experience  level.  Even  though  all  trainees  improved  between  the  training  and  testing  scenarios, 
the  degree  of  improvement  was  least  for  the  most  experienced  persons  and  moderate  for  the 
persons  with  1-3  years  of  experience.  The  least  experienced  persons  improved  the  most, 
exceeding  the  performance  of  both  the  other  groups  during  the  testing  scenario.  In  one  sense, 
this  is  not  surprising  because  the  skill  of  using  range  markers  to  determine  own  ship’s  position 
across  the  track  is  a  skill  that  is  very  specific  to  harbor  ship  handling,  especially  shallow,  narrow 
channels  like  Kings  Bay.  The  inexperienced  trainees  had  heard  lectures  on  the  use  of  range 
markers.  Some  had  even  been  through  range  marker  exercises  in  small  boats.  However,  these 
experiences  pale  before  the  experience  of  commanding  a  virtual  ship  through  a  realistic  harbor  in 
real  time.  The  experienced  trainees,  although  showing  improvement,  already  knew  how  to  use 
range  markers  and  did  not  demonstrate  as  much  improvement  on  this  skill  as  did  the 
inexperienced  trainees.  Some  of  the  experienced  trainees  might  also  have  been  trained  in 
northern  harbors  (e.g..  New  London),  which  do  not  demand  the  use  of  ranges.  These  individuals 
may  have  relied  on  the  appearance  of  their  position  between  buoys  to  judge  their  location  across 
the  track  rather  dian  using  the  ranges. 

Another  explanation  for  the  lesser  improvement  among  the  experienced  trainees  is  that  it  may 
be  a  result  of  the  more  experienced  people  not  giving  as  much  effort  as  the  least  experienced.  It 
is  also  an  indication  that  there  were  some  fidelity  problems  with  the  visual  representation  of  the 
ranges.  Several  of  the  experienced  trainees  pointed  out  that  the  representation  of  the  range 
markers  was  not  realistic  enough  and  commented  that  the  ranges  should  have  been  easier  to  find 
with  the  naked  eye.  Several  also  complained  that  it  was  difficult  to  differentiate  between  the 
range  markers  and  the  background  textures  because  of  insufficient  contrast.  Their  experiences  in 
the  real  world  (knowing  how  objects  actually  appear)  may  have  worked  against  their 
performance  in  the  virtual  world,  leading  fiiem  to  give  up  on  ranges  that  were  too  difficult  to 
find. 

Adjustments  in  the  size  of  the  range  markers,  when  viewed  without  the  binoculars,  may  be 
required  and  care  must  be  taken  to  ensure  that  the  range  markers  stand  out  firom  the  background 
as  they  do  in  the  real  world.  The  trainee  should  not  have  to  put  in  more  effort  to  find  range 
markers  than  he  would  in  the  real  world.  In  this  case,  the  more  experienced  trainees  may  not 
have  put  in  this  extra  effort.  Nevertheless,  VESUB  has  demonstrated  that  it  does  provide 
effective  training  on  the  use  of  range  markers. 

Emergency  Procedures 

During  a  harbor  transit,  a  variety  of  emergencies  can  occur.  The  OOD  must  be  able  to 
quickly  and  effectively  deal  with  these  emergencies  to  avoid  loss  of  expensive  equipment  and 
possibly  loss  of  life.  Training  on  emergency  tasks  may  be  the  most  important  contribution  that 
VE  and  other  simulation  technologies  can  provide.  VE  allows  trainees  to  experience 
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emergencies  in  real  time  and  requires  them  to  exercise  the  skills  necessary  to  deal  with  these 
emergencies  without  putting  themselves,  their  crew,  or  their  ship  in  danger. 

Two  emergency  events  occurred  during  the  scenarios:  a  man  overboard  and  a  yellow 
sounding  (a  report  of  only  5  feet  of  water  beneath  the  keel).  Two  variables  were  used  to  measure 
performance  on  the  man  overboard  event  (man  overboard  reaction  time  and  man  overboard 
commands)  and  one  variable  (yellow  sounding  commands)  was  used  to  measure  performance  on 
the  yellow  soimding  event. 

Man  Overboard  Reaction  Time.  The  most  important  action  that  the  OOD  may  take  on  the 
surface  is  saving  the  life  of  a  man  who  has  fallen  overboard.  The  ship  ’s  control  party  has  several 
immediate  actions  that  must  be  carried  out  in  this  emergency  and  it  is  the  responsibility  of  the 
OOD  to  ensure  that  they  have  been  accomplished.  This  is  done  through  the  issuance  of  orders 
and  through  verification  of  each  step.  It  is  critical  that  the  OOD  issue  these  commands  promptly 
and  correctly.  If  the  OOD  hesitates  too  long  before  issuing  the  commands,  serious  injury  or 
death  could  result. 

This  variable  measured  the  time  between  the  beginning  of  the  man  overboard  event  (i.e.,  the 
trainees  hears  “man  overboard”)  and  the  time  the  trainee  issued  the  first  command  to  deal  with 
the  emergency.  Cognitive  components  1 1C  (Understanding  When  and  How  to  Take  Corrective 
Actions)  and  12C  (Understanding  Effective  Communication  Procedures)  are  the  most  relevant  to 
this  variable.  Perceptual  component  8P  (Detecting  and  Filtering  Communications)  and  cognitive 
component  6C  (Understanding  Relative  Motion)  also  contributed  to  the  skill  requirements 
measured  by  this  variable. 

All  trainees  showed  significant  improvement  on  their  reaction  time  during  this  emergency 
event.  Although  the  more  experienced  trainees  did  not  show  as  great  a  degree  of  improvement  as 
the  inexperienced  trainees,  the  overall  mean  time  to  react  to  hearing  that  a  man  had  fallen 
overboard  by  issuing  the  first  command  was  reduced  by  44%  (from  5.34  seconds  to  2.99 
seconds)  between  the  training  and  testing  scenarios.  The  reduction  of  2.35  seconds  means  that 
the  corrective  actions  began  before  the  ship  had  traveled  an  additional  16  feet  (at  12  knots). 
Although  this  is  not  a  great  distance,  a  submarine  maneuvers  slowly  and  this  could  be  the  critical 
factor  in  allowing  enough  time  to  swing  the  stem  away  from  the  man  before  he  was  caught  in  the 
screw. 

Man  Overboard  Commands.  If  the  victim  has  survived  the  fall  into  the  channel,  he  faces  an 
immediate  danger  from  the  ship’s  control  surfaces  and  the  screw.  The  OOD  must  first  give 
orders  to  stop  the  shaft  so  that  the  blades  of  the  screw  will  not  be  turning  should  they  make 
contact  with  the  man.  Then,  he  must  give  orders  to  swing  the  ship  away  from  the  man.  This  is 
accomplished  by  putting  on  a  full  mdder  in  the  direction  of  the  man.  That  is,  if  the  man  fell  off 
of  the  starboard  (right)  side,  the  OOD  orders  a  right  full  mdder  to  swing  the  stem  to  the  left 
(away  from  the  man).  If  the  man  fell  off  of  the  port  (left)  side,  the  correct  command  would  be  a 
left  full  mdder.  After  the  man  is  clear,  the  OOD  must  give  the  necessary  commands  to  keep  own 
ship  in  good  water  and  also  take  steps  to  retrieve  the  man  (e.g.,  contact  the  tug  on  bridge-to- 
bridge  radio).  These  commands  may  be  varied  slightly  depending  on  the  prevailing  conditions 
(e.g.,  less  than  a  full  mdder  might  be  used  if  the  channel  is  very  narrow). 
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This  variable  was  the  number  of  correct  commands  given  by  the  trainee  during  the  man 
overboard  emergency.  The  most  relevant  component  is  1 1C  (Understanding  When  and  How  to 
Take  Corrective  Actions),  with  lesser  contributions  from  components  6C  (Understanding 
Relative  Motion)  and  12C  (Understanding  Effective  Communication  Procedures). 

A  significant  main  effect  of  scenario  session  was  found  on  this  variable  showing  an 
improvement  of  29%  on  the  use  of  correct  commands  during  the  man  overboard  event  for  all 
trainees.  There  was  no  significant  interaction  with  experience  level,  although  a  slightly  greater 
degree  of  improvement  was  found  with  the  less  experienced  trainees.  Data  on  this  variable 
demonstrate  once  again  that  the  VESUB  simulation  provides  the  context  to  both  initially  train 
and  also  to  refresh  critical  skills. 

Yellow  Sounding.  Yellow  or  red  soundings  are  established  as  warning  signals  which  indicate 
that  the  depth  of  water  beneath  the  keel  of  the  ship  is  too  shallow  for  safe  operation  and  require 
immediate  corrective  actions.  The  yellow  sounding  is  a  caution  that  the  ship  may  be  moving  into 
danger.  The  red  sounding  indicates  that  the  ship  is  in  imminent  danger.  The  depth  at  which  a 
yellow  or  red  sounding  occurs  is  set  by  the  Navigator  and  approved  by  the  Commanding  Officer. 
In  our  scenarios,  yellow  soundings  were  set  at  five  feet  and  red  soundings  at  three  feet.  These 
values  were  communicated  to  the  trainees  during  the  prebriefs  for  each  scenario. 

This  variable  was  the  total  number  of  correct  actions  taken  by  the  trainee  during  the  yellow 
soimding  event.  Some  of  the  actions  that  should  have  been  taken  by  the  trainee  included: 
slowing  own  ship’s  headway;  fixing  ship’s  position;  confirming  direction  to  good  water; 
maneuvering  to  seek  the  center  of  the  channel  or  best  water;  sighting  navigation  aids;  and 
reporting  to  the  CO. 

Perceptual  component  IP  (Locating  and  Identifying  Navigation  Aids)  and  cognitive 
component  1 1C  (Understanding  When  and  How  to  Take  Corrective  Actions)  are  the  most 
relevant  for  this  variable.  Perceptual  components  2P  (Judging  Distance)  and  8P  (Detecting  and 
Filtering  Communications)  and  cognitive  components  4C  (Understanding  the  Effects  of  Tides, 
Currents,  Wind,  and  Seas),  IOC  (Understanding  Correct  Operation  of  Ship’s  Systems),  and  12C 
(Understanding  Effective  Communication  Procedures)  are  also  associated  with  this  variable. 

Between  the  training  and  testing  scenarios,  a  40%  improvement  was  demonstrated  by  all 
trainees  in  the  actions  taken  during  the  yellow  sounding  event.  Although  the  most  experienced 
trainees  started  with  a  higher  skill  level  than  the  least  experienced  trainees,  they  also 
demonstrated  a  higher  skill  level  during  the  testing  scenario.  This  indicates  that  VESUB 
provides  effective  refresher  training  for  the  experienced  person  on  this  task.  It  also  indicates  that 
the  skill  levels  of  the  inexperienced  trainees  could  probably  be  increased  even  more  with 
additional  time  in  VESUB. 

Incorrect  Report  Recognition 

Although  the  OOD  is  in  charge,  he  constantly  receives  information  from  other  members  of  the 
navigation  team  to  help  make  decisions.  It  is  very  important  that  the  OOD  always  verify  the 
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accuracy  of  this  information  rather  than  blindly  following  the  reports.  During  the  scenarios,  an 
incorrect  position  report  (left  of  track  when  actually  right  or  vice  versa)  was  given  by  the 
navigator  to  emphasize  the  importance  of  report  verification.  Only  one  variable  (Incorrect 
Position)  was  used  to  assess  improvements  in  performance  in  this  skill  area.  It  measured  the 
number  of  correct  actions  taken  by  the  trainee  after  receiving  the  incorrect  position  report  from 
the  Navigator. 

Perceptual  components  IP  (Locating  and  Identifying  Navigation  Aids)  and  8P  (Understanding 
Ship’s  Operation  Under  Harbor  Directives)  and  cognitive  component  1 1C  (Understanding  When 
and  How  to  Take  Corrective  Actions)  are  the  most  relevant  to  this  variable.  Other  components 
that  also  contribute  to  the  skills  measured  with  this  variable  are;  2P  (Judging  Distance)  and  12C 
(Understanding  Effective  Communication  Procedures). 

The  significant  main  effect  for  scenario  session  showed  that  all  trainees  improved  on  their 
ability  to  recognize  the  incorrect  position  report.  This  is  an  indication  of  an  improvement  in  the 
trainees’  situation  awareness.  The  trainees  have  become  more  aware  of  the  possibility  of 
mistakes  during  interactions  with  the  Navigation  Team  and  the  possible  consequences  of  these 
mistakes. 

Although  differential  degrees  of  improvement  were  shown  by  persons  with  different 
experience  levels,  there  was  not  enough  difference  to  achieve  a  significant  interaction  with 
experience.  Once  again,  VESUB  has  been  shown  to  provide  both  initial  and  refresher  training. 

Communications 


The  OOD  does  not  physically  control  the  ship.  He  must  issue  commands  to  the  navigation 
team  in  a  timely  and  accurate  manner  and  also  verify  that  these  commands  have  been  correctly 
heard  and  that  the  correct  actions  are  being  taken.  Commimications  with  the  navigation  team  is, 
therefore,  one  of  the  most  critical  skill  areas  for  OOD  ship  handling.  Three  variables  were  used 
to  assess  the  trainees’  improvements  in  communications  skills:  commands  to  get  own  ship 
imderway,  use  of  station  identifiers,  and  acknowledging  reports. 

Commands  to  Get  Own  Ship  Underway.  This  variable  was  the  ratio  of  the  total  number  of 
correct  commands  the  trainee  gave  when  getting  the  ship  underway  to  the  total  number  of 
possible  correct  commands. 

Cognitive  components  IOC  (Understanding  Correct  Operation  of  Ship’s  Systems)  and  12C 
(Understanding  Effective  Communication  Procedures)  are  the  most  relevant  to  this  variable. 
Perceptual  component  5P  (Sense  of  Ship’s  Responsiveness)  and  cognitive  component  4C 
(Understanding  the  Effects  of  Tides,  Currents,  Wind,  and  Seas)  also  contribute  to  the  skills 
measured  by  this  variable. 

There  was  no  significant  main  effect  of  scenario.  Although  a  probability  of  .04  was  achieved, 
the  use  of  a  .01  significance  level  was  the  established  criterion  for  this  experiment.  Nevertheless, 
trainees  did  show  some  improvement  on  this  skill  between  the  training  and  the  testing  scenarios. 
The  lack  of  an  interaction  with  experience  level  showed  that  some  learning  occurred  for  all 


64 


Technical  Report  98-003 


trainees.  However,  there  was  a  main  effect  for  experience  level.  The  most  experienced  trainees 
(over  three  years)  started  with  lower  levels  of  performance  in  the  training  scenario  and 
demonstrated  very  little  improvement  on  these  skills.  This  may  be  another  instance  of  the 
experienced  persons  not  taking  the  scenarios  as  seriously  as  they  could  have. 

Acknowledging  Reports.  The  OOD  must  acknowledge  each  report  received  from  another 
member  of  the  navigation  team  to  let  them  know  he  has  heard  the  report.  This  variable,  was  the 
ratio  of  the  total  number  of  reports  that  were  acknowledged  by  the  subject  to  the  total  niunber  of 
reports  given. 

The  most  relevant  perceptual  and  cognitive  components  to  this  variable  are  8P  (Detecting  and 
Filtering  Communications)  and  12C  (Understanding  Effective  Communication  Procedures). 
Cognitive  component  IOC  (Understanding  Correct  Operations  of  Ship’s  Systems)  is  also 
involved  in  these  skills. 

No  significant  main  effects  or  interactions  were  foxmd  for  this  variable.  This  may  show  that 
the  measure  lacked  sensitivity  or  it  was  not  a  good  indicator  of  communications  skills.  It  may 
also  be  the  case  that  all  trainees  already  fully  understood  the  importance  of  acknowledging 
reports  so  that  the  training  had  no  measurable  effect  on  their  performance. 

Use  of  Station  Identifiers.  The  Doctrine  for  Submarine  Interior  Communications  (1995.  p. 
4-1)  instruction  states  that  each  transmission  consists  of  three  parts:  (a)  Station  called  identifier 
(recipient);  (b)  Station  calling  identifier  (originator);  and  (c)  Text.  This  means  that  each  time  the 
OOD  issues  a  command,  he  must  identify  to  whom  he  is  speaking  and  then  identify  himself, 
prior  to  the  command.  Although  this  “by  the  book”  approach  is  not  always  required  (e.g.,  when 
communicating  to  the  Helm  for  rudder  commands,  p.  4-12),  it  was  determined  that  for  the 
piuposes  of  the  VESUB  TEE  station  identifiers  would  always  be  required.  The  voice  recognition 
system  was  programmed  to  give  a  “Say  again.  Sir”  each  time  the  trainee  forgot  to  use  station 
identifiers. 

This  variable  measured  the  number  of  times  the  trainee  did  not  use  station  identifiers.  A 
significant  main  effect  of  scenario  session  demonstrated  improvement  on  this  skill  for  all 
trainees.  Although  there  was  no  interaction  with  experience  level,  the  more  experienced  trainees 
started  with  better  scores  (e.g.,  fewer  mistakes)  and  also  finished  with  better  scores  than  the  least 
experienced  trainees.  This  indicates  that  refresher  training  was  successful  for  the  experienced 
trainees  and  also  that  additional  improvements  should  be  expected  for  the  inexperienced  trainees. 

Rules  of  the  Road 


While  operating  in  a  harbor,  the  OOD  will  often  have  to  interact  with  other  vessels.  To  ensure 
safety  during  these  interactions,  the  U.S.  Coast  Guard  has  established  navigation  rules  of  the  road 
to  which  all  vessels  must  abide  (U.S.  Department  of  Transportation,  1990)  .  The  OOD  must 
understand  and  follow  these  rules  of  the  road  each  time  he  encounters  another  vessel.  In  our 
scenarios,  a  crossing  situation  (Rule  15,  p.  31)  with  a  ferry  boat  was  programmed  to  force  the 
trainee  to  follow  the  correct  rules  of  the  road.  In  this  case,  the  ferry  boat  was  crossing  from  the 
right  (starboard)  to  the  left  (port)  of  OAvn  ship.  This  meant  that  the  ferry  was  the  “stand-on 
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vessel”  which  allowed  it  to  “keep  her  course  and  speed”  (Rule  17,  p.  33)  and  the  own  ship  was 
the  “give-way  vessel”  which  meant  it  had  to  “take  early  and  substantial  action  to  keep  well  clear” 
of  the  other  vessel  (Rule  19,  p.  33).  The  trainee  had  to  contact  the  ferry  to  determine  its 
intentions  and  take  all  other  actions  necessary  to  avoid  a  collision.  As  the  event  progressed,  an 
agreement  was  reached  to  conduct  a  port-to-port  passage.  The  trainee  was  required  to  maneuver 
own  ship  to  the  right  side  of  the  charmel  and  slow  enough  to  minimize  wake  damage  to  the  small 
craft.  One  variable  was  used  to  assess  performance  improvements  in  rules  of  the  road  skills. 

Ferry  Passage.  This  variable  was  the  total  number  of  correct  commands  given  during  the 
crossing  and  port-to-port  passage  event.  A  large  number  of  perceptual  and  cognitive  components 
of  “Seaman’s  Eye”  are  necessary  to  support  the  decision  making  requirements  of  this  complex 
task.  The  most  relevant  perceptual  component  for  this  variable  is  4P  (Locating,  Identifying,  and 
Avoiding  Obstacles).  Three  cognitive  components  are  most  closely  associated  with  this  variable: 
5C  (Understanding  Rules  of  the  Road);  8C  (Understanding  Ship’s  Operation  Under  Harbor 
Directives);  and  12C  (Understanding  Effective  Communication  Procedures).  Three  additional 
perceptual  and  five  cognitive  variables  support  this  task  to  a  lesser  degree:  2P  (Judging 
Distance);  5P  (Sense  of  Ship’s  Responsiveness);  8P  (Detecting  and  Filtering  Communications); 
2C  (Understanding  Relative  Size  and  Height/Range  Relationships,  and  Angle  on  the  Bow);  6C 
(Understanding  Relative  Motion);  7C  (Understanding  Methods  to  Differentiate  and  Prioritize 
Traffic  Contacts);  IOC  (Understanding  Correct  Operation  of  Ship’s  Systems);  and  IIC 
(Understanding  When  and  How  to  Take  Corrective  Actions). 

Between  the  training  and  testing  scenarios,  significant  improvement  on  this  variable  was 
demonstrated  by  trainees  at  all  levels  of  experience.  Trainees  improved  on  the  number  of  correct 
commands  issued  while  dealing  with  the  crossing  situation  and  port-to-port  passage  of  the  ferry 
boat.  These  are  especially  important  skills  because  accidents  between  two  vessels  can  cause 
extensive  damage  and  possible  loss  of  life.  The  ability  to  practice  these  types  of  events  in  a  safe 
simulated  environment  is  one  of  the  most  important  benefits  available  with  VESUB. 

EXPERT  OPINIONS 

In  addition  to  collecting  data  to  demonstrate  the  training  effectiveness  of  the  VESUB  system, 
one  of  the  major  goals  of  the  TEE  was  to  collect  expert  opinions  on  the  design  and  use  of 
VESUB  and  VE  technologies.  Future  applications  of  VE  training  systems  depend  on  the  level  of 
support  obtained  from  users  of  VESUB  and  other  demonstration  systems.  Furthermore,  since 
operational  VESUB  systems  will  be  used  in  Navy  training  facilities,  the  opinions  of  instructors, 
trainees,  and  fleet  experts  can  help  determine  how  VE  training  technologies  should  be  inserted 
into  Navy  curriculums.  Opinions  were  collected  fi'om  trainees  after  the  testing  scenario  and  also 
firom  observers  who  did  not  participate  in  the  full  TEE. 

Trainees’  Opinions 

After  completion  of  the  final  “Comfort”  questionnaire,  each  trainee  filled  out  an  opinion 
questionnaire  (Appendix  G).  The  questionnaire  included  eight  forced-choice  questions  and  one 
open-ended  question.  The  forced-choice  questions  used  a  five  level  Likert  Scale  with  the  choices 
“strongly  agree,”  “agree,”  “neutral,”  “disagree,”  and  “strongly  disagree.” 
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Learning  Experience  in  VESUB.  Questions  One  and  Two  asked  about  the  trainee’s 
learning  experience  in  VESUB.  Question  One  asked  if  the  trainee  agreed  that  his  time  in 
VESUB  was  a  positive  learning  experience.  Almost  all  trainees  (97.7%)  either  “strongly  agreed” 
or  “agreed”  that  tiiey  learned  in  VESUB.  No  one  disagreed.  This  is  a  strong  indication  of  the 
positive  training  potential  of  VESUB.  Even  though  this  was  a  demonstration  system,  with  only  a 
minimal  curriculum,  the  trainees  felt  they  had  learned  meaningful  skills. 

Question  Two  asked  if  VESUB  increased  the  trainee’s  confidence  in  his  ship  handling  skills. 
Almost  93%  of  the  trainees  were  more  confident  in  their  ship  handling  skills  after  their 
experience  in  VESUB. ,  This  increased  sense  of  confidence  will  carry  over  to  their  live  bridge 
experiences,  helping  them  obtain  more  benefit  fi’om  the  on-the-job  training. 

Head-Mounted  Display.  The  next  two  questions  concerned  the  fit  and  adjustment  of  the 
HMD.  Question  Three  asked  if  the  trainee  could  fully  adjust  the  HMD  to  his  specific  visual 
needs.  Over  73%  of  the  trainees  could  adjust  the  HMD  to  fit  their  head,  but  12%  could  not. 
Making  the  focus  adjustments  easier  to  use  would  improve  the  design  of  the  HMD. 

Question  Four  asked  about  the  comfort  adjustment  of  the  HMD.  Although  90%  had  no 
comfort  problems,  almost  10%  did.  Most  of  these  problems  involved  the  adjustment  range  for 
head  size  and  the  padding  on  the  top  of  the  HMD.  Additional  work  needs  to  be  done  to  improve 
the  design  of  HMDs.  Reductions  in  weight  and  improvements  in  size  range  would  help. 

Types  of  Training  with  VESUB.  Questions  Five  and  Six  concerned  the  type  of  training  that 
was  most  suitable  for  VESUB.  The  fifth  question  asked  the  trainee  if  he  agreed  that  VESUB 
should  be  used  for  introductory  training.  Almost  all  trainees  responded  positively  to  the  idea  of 
using  VESUB  for  introductory  training.  In  many  cases,  VESUB  could  be  the  first  time  a  trainee 
is  on  the  bridge.  It  can  provide  a  realistic  exposure  to  the  cues  that  the  OOD  must  recognize  so 
that  he  is  better  able  to  adjust  to  his  presence  on  the  real  bridge  during  later  training.  Beyond 
that,  VESUB  provides  the  opportunity  to  practice  emergency  tasks  that  would  never  be  possible 
in  the  real  world. 

The  sixth  question  asked  if  VESUB  should  be  used  for  refi'esher  training.  Over  92%  of  the 
trainees  believe  that  VESUB  can  provide  effective  refi-esher  training.  This  is  a  particularly 
strong  recommendation,  given  that  so  many  of  the  trainees  were  highly  experienced  ship 
handlers. 

Location  of  Training.  The  seventh  forced-choice  question  dealt  with  where  training  should 
be  delivered.  It  asked  if  the  trainee  agreed  that  VESUB  should  only  be  used  in  the  classroom  for 
shore-based  training.  Although  50%  of  the  trainees  disagreed  that  VESUB  should  only  be  used 
ashore  in  the  classroom,  there  may  have  been  some  confusion  about  this  question.  It  is  the  only 
question  that  was  worded  so  that  a  positive  response  was  “disagree.”  A  few  trainees  may  not 
have  carefully  read  the  question  and  answered  “agree”  because  it  was  the  pattern  they  had 
followed  on  the  other  questions.  Nevertheless,  these  responses  indicate  that  VESUB  could  be 
used  for  training  in  other  contexts  besides  the  classroom  (e.g.,  aboard  the  ship). 
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Mission  Rehearsal.  The  last  forced-choice  question  asked  the  trainees’  opinions  on  the  use 
of  VESUB  for  mission  rehearsal  aboard  the  submarine.  This  question  might  also  have  confused 
some  trainees.  They  may  not  have  imderstood  what  was  meant  by  mission  rehearsal.  Even  so, 
over  50%  of  the  trainees  felt  that  the  system  could  be  used  to  practice  missions. 

Onen-ended  Comments.  The  open-ended  question  provided  an  opportunity  for  the  trainees 
to  state  any  suggestions  for  improving  VESUB  or  opinions  about  the  VESUB  system.  Their 
comments  were  grouped  into  several  content  areas. 

General  Reactions  to  the  System.  Trainees  were  very  positive  about  their  experiences  in 
VESUB.  Adjectives  such  as  great,  outstanding,  and  excellent  were  used  throughout  their 
comments.  Almost  all  the  trainees  thought  the  learning  potential  of  VESUB  was  very  high. 

Only  one  negative  comment  was  received  jfrom  a  trainee  who  had  to  be  removed  from  the  system 
due  to  simulator  after  effects. 

Instructional  Tools  or  Approaches.  Trainees  appreciated  the  opportunity  to  practice  in 
situations  where  no  one  was  in  danger.  They  also  liked  the  opportimity  to  obtain  incorrect 
information  (e.g.,  the  incorrect  navigator  report)  and  be  able  to  take  corrective  actions  in  a  safe 
environment. 

Voice  and  Sounds.  The  voice  recognition  system  was  recognized  as  an  integral  component 
of  the  system.  It  was  also  an  area  that  received  a  large  number  of  suggestions  for  improvement. 
The  largest  number  of  suggestions  concerned  the  ability  of  the  OOD  to  override  the  speech 
generation  system.  In  the  demonstration,  the  system  could  not  hear  the  trainee  when  it  was 
speaking  to  him.  This  became  quite  cumbersome,  especially  in  emergency  events.  The  OOD’s 
microphone  should  turn  off  the  speech  synthesis  system  any  time  it  is  activated.  The 
requirement  to  use  station  identifiers  (e.g.,  “Helm,  Bridge. . .)  was  not  appreciated  by  some 
trainees,  although  others  saw  value  in  requiring  them.  The  best  approach  would  be  to  have  the 
system  recognize  the  command  without  Ae  station  identifiers,  but  record  that  the  trainee  did  not 
use  them.  Later,  the  instructor  could  provide  feedback  to  improve  performance  in  this  area. 

Other  comments  concerned  fine  tuning  the  speed  of  both  the  voice  recognition  and  synthesis 
systems. 

Environmental  sounds  were  highly  regarded  and  several  suggestions  to  improve  their  realism 
were  made  by  the  trainees.  The  sounds  of  wind  and  waves  can  provide  speed  cues  and 
directional  sounds  can  help  the  trainee  locate  contacts  and  nearby  navigation  aids. 

Visual  Scene  and  Visual  System.  The  most  important  suggestion  in  this  area  is  to  replace 
the  voice-activated  binocular  function  with  a  physical  switch  on  the  HMD.  The  voice  command 
for  binoculars  was  included  in  the  original  feasibility  demonstration  system  and  carried  over  to 
VESUB.  It  seemed  a  good  idea  until  the  ship  handling  vocabulary  was  fully  implemented.  The 
voice  activation  for  binoculars  became  too  cumbersome  when  trying  to  issue  other  commands. 
The  operational  VESUB  system  should  include  a  switch  on  the  HMD  that  will  activate  the 
binoculars  only  when  held  down  and  revert  to  naked  eye  view  when  released. 
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Several  suggestions  were  made  to  improve  the  appearance  of  the  visual  scene.  The  contrast 
between  navigation  aids  and  background  textures  was  seen  as  an  area  where  improvement  was 
needed,  as  was  the  appearance  of  contacts.  Several  observers  mentioned  that  the  distance  to 
contacts  could  be  more  accurately  estimated  if  the  database  provided  curvature  of  the  earth.  In 
the  real  world,  contacts  at  large  (hstances  fall  below  the  horizon  and  become  “hull  down,”  where 
only  their  masts  and  upper  decks  are  visible.  In  our  databases,  the  earth  is  flat  and  the  contacts 
display  their  entire  structure  at  all  ranges  providing  inaccurate  distance  cues. 

Improvements  were  also  suggested  to  make  the  bow  wave  and  stem  wake  more  realistic  to 
provide  more  accurate  speed  cues.  As  the  ship  increases  speed,  the  bow  wave  should  move 
toward  the  sail  and  become  more  pronounced  and  blend  with  the  stem  wake.  The  stem  wake 
should  extend  much  farther  (up  to  several  miles)  than  in  the  ciirrent  display. 

Submarine  Model.  One  trainee  thought  the  submarine  responded  too  quickly  to  speed  (bell) 
orders.  Most,  however,  thought  the  models  were  accxirate  enough  for  training  purposes  with  the 
exception  of  the  way  own  ship  bounced  over  the  waves  rather  than  pushing  under  them.  Several 
suggested  that  collision  detection  should  also  be  included  in  the  model.  This  would  provide  an 
indication  when  the  submarine  touched  a  contact,  the  bottom,  navigation  aids,  or  piers. 

Use  of  the  System.  Almost  all  trainees  agreed  that  the  system  was  useful  for  introductory  and 
refresher  training.  A  couple  of  suggestions  were  made  to  allow  all  members  of  the  Navigation 
Team  to  train  in  VESUB.  This  would  give  each  team  member  a  better  understanding  of  the 
requirements  and  information  needs  of  the  OOD. 

Head-Mounted  Display.  The  TEE  was  an  opportunity  to  allow  trainees  to  wear  the  HMD 
for  an  extended  period  of  time  (almost  3  hours).  Some  trainees  suggested  additional  padding  to 
make  the  HMD  more  comfortable.  Others  suggested  better  fit  and  focus  adjustment  capabilities. 
A  wider  field  of  view  was  desired  by  almost  all  trainees. 

Haptic  Capabilities.  One  trainee  suggested  the  inclusion  of  a  virtual  hand  in  the  visual  scene 
to  assist  the  OOD  in  taking  bearings  or  enabling  him  to  touch  virtual  bridge  equipment.  Several 
other  trainees  provided  similar  verbal  comments.  This  would  require  a  separate  hand  tracker,  but 
should  not  be  too  difficult  and  would  improve  the  realism  of  the  system. 

Observers’  Opinions 

Observers  of  the  TEE  filled  out  an  opinion  questionnaire  (Appendix  H).  It  consisted  of  six 
forced-choice  and  one  open-ended  question. 

Training  Potential  of  VESUB.  The  first  two  questions  asked  the  observers’  opinions  about 
the  training  potential  of  VESUB.  Question  One  asked  if  the  observers  agreed  that  VESUB 
provided  a  positive  learning  experience.  All  of  the  observers  answered  positively  on  this 
question,  demonstrating  that  even  without  extensive  experience  in  VESUB,  they  recognized  its 
capabilities  to  provide  effective  instruction.  On  Question  Two,  over  90%  of  the  observers 
believed  that  experience  with  VESUB  would  improve  a  trainee’s  confidence  in  his  ship  handling 
skills. 
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Types  of  Training.  Whether  VESUB  should  be  used  for  introductory  training,  refresher 
training,  or  both  were  addressed  in  Questions  Three  and  Four.  Almost  all  (98%)  of  the  observers 
agreed  that  it  should  be  used  for  introductory  training  and  91%  thought  it  could  also  be  used  for 
refresher  training.  Data,  discussed  earlier,  showing  that  learning  occurred  with  trainees  at  all 
experience  levels,  support  the  opinions  of  the  observers  that  VESUB  should  be  used  for  both 
introductory  and  refresher  training  of  ship  handling  skills. 

Where  to  Train.  Questions  Five  and  Six  concerned  whether  VESUB  should  be  used  only  in 
the  classroom  or  deployed  aboard  ships.  Reactions  were  mixed  on  whether  it  should  only  be 
used  in  the  classroom  (54%  disagreed).  However,  the  wording  of  this  question  called  for  a 
reversal  in  the  pattern  of  responses,  possibly  confusing  some  respondents. 

Although  their  responses  were  mixed,  over  half  of  the  observers  agreed  that  VESUB  could  be 
deployed  and  used  for  mission  rehearsal.  Many  additional  issues  must  be  resolved  before 
VESUB  can  be  used  in  this  manner.  For  example,  the  size  of  the  computer  hardware  must  be 
reduced  and  some  method  to  download  databases  or  to  maintain  a  database  library  would  be 
required.  Additional  research  will  eventually  provide  this  capability. 

Open-ended  Suggestions.  The  observers  provided  munerous  comments  in  the  open-ended 
section  of  the  questionnaire.  Their  suggestions  were  grouped  into  several  content  areas. 

General  Reactions  to  the  System.  The  observers’  general  reactions  to  the  system  were  very 
positive.  The  current  configuration  was  viewed  as  an  excellent  first  step  with  great  potential. 

System  Expansions.  Even  though  they  recognized  the  potential  of  VESUB,  the  observers 
suggested  ways  to  expand  the  capabilities  of  the  system.  An  open  ocean  database  would  allow 
training  of  tasks  such  as  underway  replenishment  and  persoimel  transfers.  Observers  also 
mentioned  the  need  for  training  in  mooring  and  docking  procedures.  These  tasks  are  difficult 
and  have  the  potential  for  damage  to  own  ship  and  assist  tugs.  A  mooring  and  docking  capability 
was  minimally  available  in  the  configmation  of  the  demonstration  system,  but  needs 
considerable  improvement  in  the  operational  systems. 

Interfacing  VESUB  with  the  existing  Submarine  Piloting  and  Navigation  (SPAN)  trainer  was 
an  original  goal  of  the  research  effort.  Unfortunately,  budget  and  schedule  constraints  did  not 
allow  this.  Observers  suggested  that  integration  of  OOD  training  with  other  team  members 
(maneuvering  watch  and  piloting  teams)  would  be  highly  beneficial.  The  operational  VESUB 
systems  will  need  to  be  developed  to  include  the  capabilities  to  interface  with  other  team 
members.  It  was  also  suggested  that  VE  technologies  could  be  used  for  other  tasks,  such  as 
maintenance  training. 

Instructional  Tools.  Highlighting  of  channel  boundaries  and  centerline  was  available  in  the 
demonstration  system.  Most  people  thought  this  was  a  great  idea.  However,  both  sides  of  the 
channel  were  represented  with  green  lines.  A  logical  suggestion  was  to  make  the  red  buoy  side 
red  and  the  green  buoy  side  green. 
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Voice  and  Sounds.  The  voice  and  sound  system  was  seen  as  very  important  for  successful 
training.  Several  suggestions  were  made  to  improve  these  capabilities.  The  most  useful 
suggestion  was  to  implement  the  voice  recognition  so  it  rmderstands  the  trainee,  even  when  the 
voice  synthesis  system  is  speaking.  This  is  critical  in  fast-paced,  emergency  situations  when 
every  second  counts.  The  system  was  implemented  to  require  station  identifiers  (e.g.,  “Helm, 
Bridge”)  before  each  command.  Although  this  is  “by  the  book,”  the  system  needs  to  be  able  to 
understand,  even  without  the  station  identifiers.  In  emergency  situations,  it  is  not  realistic  to 
have  the  system  say  “Say  Again,  Sir”  when  station  identifiers  are  forgotten.  It  needs  to  record 
this  mistake,  but  still  follow  the  command.  The  binocular  command  was  also  seen  as 
cumbersome  in  emergency  situations.  Use  of  a  mechanical  switch  to  turn  binoculars  on  and  off 
will  alleviate  this  problem. 

Visual  Scene  and  Visual  System.  Several  suggestions  were  provided  to  make  the  visual 
scene  more  realistic.  Adding  additional  bridge  equipment  (Radar,  GPS,  a  means  to  display 
contact  positions)  are  all  features  that  should  be  considered  in  the  operational  system,  depending 
on  their  availability  on  the  bridge  of  the  selected  ship  class.  Technological  innovations  will 
allow  the  visual  fidelity  of  the  environment,  the  visual  refi-esh  rate,  and  field  of  view  to  be 
improved. 

Virtual  Crew.  The  addition  of  the  Contact  Coordination  was  seen  as  essential  both  fi-om  the 
perspective  of  the  trainee  and  the  instructor.  Playing  the  role  of  the  Contact  Coordinator  requires 
far  too  many  instructor  resources  that  could  be  better  used  observing  and  coaching  the  trainee. 
The  Contact  Coordinator  is  the  most  complex  virtual  crewman  since  the  system  must  deal  with  a 
higher  level  of  free  speech  than  with  other  stations.  Development  of  this  capability  will  require 
innovative  modeling  of  the  Contact  Coordinator’s  job  and  improved  logic  in  the  voice 
recognition  system. 

Addition  of  Hantic  Capabilities.  The  addition  of  haptic  (touch  and  feel)  capabilities  was 
suggested  by  several  observers.  This  will  allow  the  trainee  to  interact  more  realistically  with 
bridge  equipment,  such  as  the  compass  alidade  and  the  microphone.  Even  the  minimal  addition 
of  a  virtual  body  in  the  trainee’s  visual  scene  could  improve  his  spacial  orientation  and  probably 
enhance  training.  Additional  research  is  needed  to  determine  how  existing  and  emerging  haptic 
capabilities  can  be  incorporated  in  future  VR  systems. 
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CONCLUSIONS  AND  RECOMMENDATIONS 

The  VESUB  R&D  technology  demonstration  system  is  one  of  the  first  VE  training  systems  to 
be  developed  at  a  comprehensive  enough  level  to  be  evaluated  at  training  facilities.  Even  so,  it  is 
not  a  complete  training  system.  Although  learning  was  demonstrated  on  quite  a  few  critical 
variables  in  important  skill  areas,  many  ship  handling  skills  were  not  evaluated  in  the  VESUB 
demonstration  system.  In  some  cases  this  is  because  the  capability  to  train  these  skills  was  not 
available  in  the  VESUB  demonstration  system.  In  other  cases,  time  and  budget  constraints  did 
not  allow  the  evaluation  of  these  skills  areas  in  the  TEE.  Nevertheless,  the  results  of  the  TEE  do 
point  to  several  important  conclusions  and  recommendations: 

♦  VESUB  provides  effective  ship  handling  training 

♦  All  experience  levels  can  benefit  firom  VESUB  training 

♦  Event-based  curriculum  development  is  recommended 

♦  Instructor  training  is  essential 

♦  VESUB  is  the  first  step  toward  SPAN  2000 

♦  Improvement  and  expansion  of  the  virtual  crew  is  required 

♦  Team  training  can  be  improved  with  the  use  of  simulated  team  members 

VESUB  PROVIDES  EFFECTIVE  SHIP  HANDLING  TRAINING 

Data  firom  the  TEE  on  eleven  of  fifteen  variables  showed  significant  learning  in  a  variety  of 
ship  handling  skill  areas.  It  can  be  said  with  confidence  that  VESUB  technologies  can  provide 
effective  training.  However,  a  training  system  is  far  more  than  just  technologies.  It  can  not  be 
stated  too  strongly  that  a  training  system  will  only  be  effective  if  it  is  used  correctly.  Care  must 
be  taken  to  implement  the  technologies  in  VESUB  in  a  manner  that  is  consistent  with  known 
learning  principles. 

ALL  EXPERIENCE  LEVELS  CAN  BENEFIT  FROM  VESUB  TRAINING 

Data  on  almost  every  variable  showed  that  trainees  with  every  level  of  experience  can  benefit 
firom  training  in  VESUB.  Very  inexperienced  trainees,  those  that  have  never  been  on  the  bridge 
of  a  submarine,  can  experience  the  feelings  and  events  of  a  real-time,  high  fidelity  simulation 
that  provides  most  of  the  cues  that  must  be  dealt  with  in  the  real  world.  Simulations  will  not 
replace  on-the-job  training  on  the  submarine  bridge,  but  the  inexperienced  JO  can  bring  what  he 
has  learned  in  the  VE  to  the  bridge  and,  therefore,  obtain  more  benefits  firom  the  on-the-job 
training  than  otherwise.  Fmrthermore,  he  can  obtain  training  on  tasks  that  are  not  always 
available  or  possible  on  the  submarine  due  to  time  and  safety  constraints. 

The  experienced  trainee,  perhaps  because  he  has  been  on  a  shore  assignment,  can  forget 
critical  elements  of  ship  handling  tasks.  The  TEE  demonstrated  that  even  persons  with  over  10 
years  experience  can  improve  their  ship  handling  skills.  Prospective  COs  and  XOs  can  be  more 
confident  in  their  ability  to  train  and  evaluate  their  JOs  if  their  own  skills  are  firesh  in  their  minds. 
VE  training  systems,  like  VESUB,  will  eventually  be  less  costly  so  they  can  be  made  available  to 
persons  at  all  levels  of  experience. 
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EVENT-BASED  CURRICULUM  DEVELOPMENT  IS  RECOMMENDED 

Any  training  system  will  only  be  effective  if  it  is  used  in  a  well  designed  instructional 
curriculum.  For  the  purposes  of  the  TEE,  an  abbreviated  curriculum  was  developed  to  target 
several  important  skill  areas.  An  event-based  approach  was  used  to  design  scenarios  that  allowed 
the  trainees  to  experience  the  critical  cues  required  for  ship  handling  tasks  and  to  practice  the 
skills  required  to  successfully  accomplish  them.  The  operational  VESUB  systems  will  provide 
effective  training  if  a  similar  approach  is  followed.  Curriculum  developers  (or  instructors)  must 
target  the  critical  skills  required  to  support  all  elements  of  ship  handling  (e.g.,  the  perceptual  and 
cognitive  components  of  “Seaman’s  Eye”).  Then,  they  must  provide  the  scenario  events  and  the 
instructional  interventions  which  will  allow  trainees  to  improve  these  skills. 

INSTRUCTOR  TRAINING  IS  ESSENTIAL 

We  can  not  expect  Navy  instructors  to  be  training  experts  unless  they  are  provided  with  the 
necessary  support  to  reach  this  goal.  A  well  designed  instructor/operator  station  can  help  by 
making  the  instructor’s  job  easier  (see  Hays,  et  al.,  1997).  This  alone  is  not  enough.  Instructors 
need  to  be  trained  on  effective  instructional  techniques.  With  new  training  technologies  come 
new  responsibilities  for  the  instructor.  They  heed  to  understand  the  capabilities  of  the  training 
system  and  how  to  use  these  capabilities  to  achieve  the  best  results.  On-line  help  and  advice  can 
be  a  part  of  this  training  as  can  a  well  designed  training  system  instructor’s  manual.  However, 
these  are  not  sufficient.  Instructors  must  be  trained  to  understand  that  learning  is  a  multistep 
process.  Each  trainee  must  receive  instruction,  be  allowed  to  practice  relevant  skills,  and  be 
given  constructive  performance  feedback.  If  instructors  learn  how  to  excel  in  all  these  areas, 
trainees  will  learn  more  effectively. 

VESUB  IS  THE  FIRST  STEP  TOWARD  SPAN  2000 

One  of  the  original  goals  of  the  VESUB  R&D  project  was  to  interface  with  the  existing 
Submarine  Piloting  and  Navigation  (SPAN)  trainer.  All  the  other  members  of  the  Navigation 
Team,  except  the  OOD,  train  on  SPAN.  Although  some  progress  was  made  toward  this  goal,  the 
outdated  hardware  and  software  in  SPAN  made  it  too  difficult  to  interface  with  newer  VESUB 
software  with  the  resources  available  in  the  R&D  project.  Nevertheless,  it  is  important  to 
provide  a  system  that  will  support  live  training  for  the  entire  Navigation  Team.  Plans  are  in 
place  for  operational  VESUB  systems  and  also  for  future  SPAN  trainers.  It  is  strongly 
recommended  that,  rather  than  trying  to  interface  the  new  VESUBs  with  existing  SPANs,  the 
image  generation  and  geographic  location  software  from  VESUB  be  used  to  drive  the  periscope, 
radar,  and  other  equipment  in  the  future  SPAN  systems.  This  approach  will  provide  improved 
visual  capabilities  in  SPAN  and  also  bring  the  OOD  into  live  team  training  with  other  members 
of  hi  s  team. 

IMPROVEMENT  AND  EXPANSION  OF  THE  VIRTUAL  CREW 

An  important  part  of  the  VESUB  demonstration  was  the  development  of  several  virtual  crew 
members.  The  Helm,  Navigator,  and  Maneuvering  were  simulated  (“virtualized”)  so  the  OOD 
could  realistically  interact  with  them  without  the  necessity  for  the  instructor  to  play  these  roles. 
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This  capability  needs  to  be  improved  and  expanded  to  other  crew  members.  For  example,  the 
Contact  Coordinator  position  is  vital  for  providing  accurate  and  timely  information  to  the  OOD 
on  the  status  of  contacts.  Resources  did  not  allow  the  completion  of  this  virtual  position,  but  it 
should  be  included  in  the  operational  VESUB  systems. 

Even  the  positions  that  were  included  in  the  virtual  crew  need  to  be  improved.  Repeat  backs 
of  commands  need  to  be  faster  and  recorded  in  a  clearer  voice.  Recognition  of  commands  needs 
to  occur  even  without  the  use  of  station  identifiers,  although  the  system  should  record  each  time 
the  trainee  fails  to  use  them.  In  emergency  situations,  like  man  overboard,  the  Helm  would  not 
say  “Say  again.  Sir”  when  ordered  “Left  fiill  rudder.”  He  would  go  ahead  and  execute  the 
command.  The  virtual  hehn  must  also  perform  this  way. 

A  communications  log  with  a  time  tag  on  each  recognized  phrase  and  synthesized  command 
should  be  provided  as  part  of  the  performance  feedback  materials  collected  by  the  system  (see 
below).  This  log  will  provide  the  opportunity  for  the  instructor  to  coach  the  trainee  on  the  use  of 
station  identifiers  and  correction  of  other  mistakes  made  during  the  training  scenario. 

TEAM  TRAINING  WITH  SIMULATED  TEAM  MEMBERS 

The  experiences  gained  in  the  development  and  use  of  the  virtual  crew  in  VESUB  can  be  of 
great  benefit  in  the  development  and  use  of  simulated  team  members  for  both  the  operational 
VESUB  and  other  future  training  systems.  As  training  resources  are  reduced,  it  becomes  more 
and  more  important  to  have  flexible  training  options.  Since  most  military  tasks  are  done  in 
teams,  team  training  is  vital  for  successful  task  performance.  However,  team  training  can  not 
always  be  conducted  with  an  intact  team.  One  or  more  critical  team  members  may  be  ill  or  away 
because  of  an  emergency.  The  rest  of  the  team  has  few  options.  They  may  work  around  the 
missing  team  member,  have  someone  else  play  that  role,  or  wait  until  he  or  she  returns. 

Simulated  team  members  can  allow  the  team  to  engage  in  meaningful  training  even  if  one  or 
more  team  members  are  not  available.  Alternatively,  a  team  member  in  need  of  remedial  training 
could  work  with  an  entire  simulated  team,  improving  his  or  her  skills  without  engaging  the  rest 
of  the  team.  Once  proficiency  has  been  obtained,  the  team  member  could  return  to  live  training. 

This  can  only  work  if  each  simulated  team  member  is  developed  with  enough  realism  to 
support  the  needs  of  the  rest  of  the  team.  This  means  that  each  team  position  has  to  be  modeled 
and  this  model  implemented  so  that  the  live  team  members  do  not  recognize  when  another 
persons  is  real  or  simulated.  Tasks  like  submarine  OOD  ship  handling  training  are  the  perfect 
context  for  developing  these  capabilities  because  the  vocabulary  is  relatively  fixed  and  no  face- 
to-face  communication  is  required. 
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COORDINATION 


The  VESUB  technology  demonstration  system  evolved  from  a  feasibility  demonstration 
system,  developed  under  the  Virtual  Environment  Training  Technology  project  at  NAWCTSD. 
This  feasibility  demonstration  was  used  during  the  first  year  of  the  VESUB  project  to  solicit 
inputs  from  fleet  Subject  Matter  Experts  (SMEs)  on  the  required  functionality  for  the  VESUB 
technology  demonstration  system.  During  development  and  formative  evaluations  of  VESUB, 
numerous  fleet  SMEs  provided  guidance  to  ensure  that  the  system  was  realistic  and  provided  the 
necessary  functionality  to  support  training  for  the  ship  handling  task.  Additional  guidance  on 
future  system  functionality  was  provided  by  several  instructors  from  SUBTRAFAC,  Norfolk, 
VA  and  NAVSUBSCOL,  Groton,  CT,  as  well  as  personnel  from  several  submarines  during  the 
TEE. 


The  development  of  VESUB  has  been  immeasurably  aided  by  the  support  and  guidance  of  the 
VESUB  Implementation  Planning  Group  (IPG).  The  IPG  included  members  from: 


♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 

♦ 


CNO  (N879C),  Washington,  DC 
CNET  (T2223),  Pensacola,  FL 
COMSUBLANT  (N742),  Norfolk,  VA 
COMSUBPAC,  Pearl  Harbor,  HI 
COMSUBGRP  Ten,  Kings  Bay,  GA 
NAVSUBSCOL  (N52),  Groton,  CT 
SUBTRAFAC  (Code  10),  Norfolk,  VA 
TRITRAFAC,  Kings  Bay,  GA 
TRITRAFAC,  Bangor,  WA 
SUBTRAFAC,  San  Diego,  CA 


VESUB  has  been  demonstrated  to  himdreds  of  visitors  to  the  NAWCTSD  laboratories  and  at 
two  major  conferences  (the  1997  Navy  League  Sea  Air  &  Space  Exposition  and  the  1997 
friterservice/Industry  Training  Systems  and  Education  Conference).  The  success  of  the 
integration  of  hardware  and  software  in  VESUB  has  led  to  the  use  of  several  components  in  other 
training  systems. 
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GLOSSARY  OF  TERMS,  ACRONYMS,  AND  ABBREVIATIONS 


AHD 

Ahead 

AME 

Nichols/Advanced  Marine  Engineering  (formerly  Advanced  Marine 
Enterprises),  VESUB  System  Developers 

ANAV 

Assistant  Navigator 

AOB 

Angle  on  the  Bow 

AST 

Astern 

CO 

Commanding  Officer 

COTS 

Commercial  Off  the  Shelf 

CPA 

Closest  Point  of  Approach 

GUI 

Graphical  User  Interface 

HCI 

Human-Computer  Interface 

HMD 

Head  Mounted  Display 

lALA 

International  Association  of  Lighthouse  Authorities 

lOS 

Instructor/Operator  Station 

JO 

Junior  Officer 

LADBM 

Large  Area  Database  Management 

NAVAID 

Navigation  Aid 

NAVSUBSCOL 

Naval  Submarine  School 

NAWCTSD 

Naval  Air  Warfare  Center  Training  Systems  Division 

NTDS 

Naval  Tactical  Data  System 

OOD 

Officer  of  the  Deck 

SIMGEN 

Simulation  Generation  Software  on  VESUB  and  Virtual  Ship 

SME 

Subject  Matter  Experts 

SPAN 

Submarine  Piloting  and  Navigation  Training  System 

SS 

Submarine  Qualified 

SUBTRAFAC 

Submarine  Training  Facility 

TEE 

Training  Effectiveness  Evaluation 

VE 

Virtual  Environment  (synonymous  with  VR) 

VESUB 

Virtual  Environment  for  Submarine  OOD  Ship  Handling  and  Piloting 
Training  System 

VETT 

Virtual  Environment  Training  Technology  (6.2  Research  Program  at 
NAWCTSD) 

Virtual  Ship 

AME’s  Commercial  Surface  Ship  Handling  Training  Product  (baseline  for 
VESUB) 

VR 

Virtual  Reality  (synonymous  with  VE) 

XO 

Executive  Officer 

81 


Technical  Report  98-003 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


Technical  Report  98-003 


APPENDIX  A 

VESUB  Hardware  and  Software 


HARDWARE 

Main  Computer  System:  Silicon  Graphics  Onyx  Deskside 

•  Four  RIO, 000  CPUs 

•  256  Mbytes  RAM 

•  One  Infinite  Reality  Graphics  Pipe  with  Two  Raster  Manager  Boards  (16  MB  Texture  Memory) 

•  Scene  Refresh  Rate  at  up  to  30  Hz 

•  Capable  of  Displaying  Approximately  21,000  Polygons  (fully  Z-bufFered  and  anti-ahased) 

Instnictor/Operator  Station  (lOS): 

•  Two  Sihcon  Graphics  INDY  Desktop  Computers 

Head  Mounted  Display  (HMD):  n-Vision  Datavisor  HiRes 

•  Resolution:  1280  x  1024  pixels 

•  Field  of  View:  40  degrees  horizontal  and  30  degrees  vertical  (capable  of  stereo-optics  for  up  to  70  degrees 
horizontal  view) 

Head  Tracker:  Polhemus  3  space  Fastxack  (Magnetic) 

Printer:  Epson,  Stylus  Color  800 

Sound  System: 

•  Two  Radio  Shack  SSM60  Stereo  Sound  Mixers 

•  Two  Rane  MSI  Microphone  Amplifiers 

•  Two  Radio  Shack  Dynamic  CB  Microphones,  P/N  21-1172 

•  Two  Radio  Shack  Speakers,  Cat.  No.:  40-1324 

SOFTWARE 

Visual  Scene: 

•  Models  and  Terrain  Created  Using  ModelGen2  from  Multigen,  Inc. 

•  Real-time,  Interactive  3-D  Scene  Generation  Controlled  by  SGI’s  IRIS  Perfonner 

•  Marine  Visual  Effects  Created  Using  Vega  Marine  from  Paradigm  Simulation,  Inc. 

Instructor/Operator  Station  (lOS)  Interface: 

•  lOS  Screens  Created  Using  Visual  Applications  Builder  (VAPS)  from  Virtual  Prototypes,  Inc. 

•  Windows  in  SIMGEN  and  Start-up  Screens  Created  wifii  X-Designer  Release  4.5  firom  Imperial 

Software  Technology  Limited  and  Data  Views  Corporation 

Voice  Recognition  &  Synthesis:  HARK,  developed  by  Bolt,  Beranek  &  Neuman,  Inc. 

Implemented  by  UFA,  Inc. 

Audio  Effects: 

•  Audioworks  from  Paradigm  Simulation,  Inc. 
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APPENDIX  B 

VESUB  Training  Objectives 


PERCEPTUAL 

COMPONENTS  TRAINING  OBJECTIVES 


IP.  Locating  and 
Identifying 

Navigation  Aids 

•  The  trainee  shall  be  able  to  locate  navigation  aids  when  referenced  by  the 
navigator. 

•  The  trainee  shall  be  able  to  recognize  navigation  aids  in  the  visual  field  and  relate 
them  to  the  chart. 

2P.  Judging  Distance 

•  The  trainee  shall  be  able  to  accurately  judge  distances  to:  navigation  aids,  contacts, 
and  landmarks. 

•  The  trainee  shall  be  able  to  judge  distances  relative  to  track. 

•  The  trainee  shall  be  able  to  verify  known  distances  using  environmental  cues  and 
chart  work. 

•  The  trainee  shall  be  able  to  maintain  the  ship  within  the  acceptable  limits  of  the 
channel. 

3P.  Identifying  Start  and 
Completion  of  Turns 

•  The  trainee  shall  be  able  to  determine  relative  and  true  directions  on  a  compass. 

•  The  trainee  shall  be  able  to  determine  relative  bearings  to  the  navigational  aid  to  be 
used  for  turn  bearings. 

•  The  trainee  shall  check  turn  bearings  when  turning. 

•  The  trainee  shall  be  able  to  recognize  when  the  ship  is  clear  to  turn 
fe.g.,  when  buoys  are  in  line). 

4P.  Locating, 

Identifying,  and 
Avoiding  Obstacles 

•  The  trainee  shall  look  far  enough  ahead  to  evaluate  contacts  early. 

•  The  trainee  shall  be  able  to  recognize  new  contacts  prior  to  being  informed  of  the 
contact. 

•  The  trainee  shall  be  able  to  recognize  relative  directions  and  motions. 

•  The  trainee  shall  be  able  to  locate,  identify,  classify  and  differentiate  between 
various  types  of  contacts  and  otiier  obstacles  (e.g.,  debris,  aquatic  animals). 

•  The  trainee  shall  take  early  and  effective  actions  to  avoid  obstacles  or  lessen  their 
negative  outcomes. 

5P.  Sense  of  Ship’s 
Responsiveness 

•  The  trainee  shall  understand  die  ship’s  capabilities  and  limitations,  including: 
advance  and  transfer,  speed  at  various  engine  orders,  loss  of  steerage  way, 
distance  to  stop  or  reverse  course. 

6P.  Recognizing 
Environmental 
Conditions 

•  The  trainee  shall  be  able  to  accurately  estimate:  sea  state,  cloud  cover,  direction 
and  velocity  of  current,  wind  direction  and  speed,  and  time  of  day. 

•  The  trainee  shall  be  able  to  accurately  judge  the  state  of  visibility. 

7P.  Recognizing 

Equipment  Failures 

•  The  trainee  shall  stay  alert  for  equipment  failures. 

•  The  trainee  shall  regularly  monitor  rudders,  indicators,  and  other  equipment. 

8P.  Detecting  &  Filtering 
Communications 

•  The  trainee  shall  be  able  to  recognize  communication  sources  and  proper/improper 
repeat  backs. 
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APPENDIX  B:  VESUB  Training  Objectives  (Continued) 


COGNITIVE 

COMPONENTS 

TRAINING  OBJECTIVES 

1C.  Understanding  the 
Relationship  of 

Visual  Cues  to 
Chart(s) 

•  The  trainee  shall  be  familiar  with  all  the  navigation  aids  to  be  used. 

•  The  trainee  shall  understand  how  to  read  ranges  (fore  and  aft). 

•  The  trainee  shall  understand  how  to  determine  if  the  ship  is  left/right  of  track 
versus  left/right  of  range. 

•  The  trainee  shall  imderstand  when  to  attempt  to  drive  the  ship  in  the  center  of  the 
channel. 

•  The  trainee  shall  understand  buoyage  systems  (e.g.,  lALA  “A”  and  “B”  systems). 

•  The  trainee  shall  imderstand  the  inaccuracy  of  buoys. 

•  The  trainee  shall  understand  the  accuracy/inaccuracy  of  Fix  information. 

2C.  Understanding 
Relative  Size  and 
Height/  Range 
Relationships,  and 
Angle  on  the  Bow 

•  The  trainee  shall  understand  how  to  determine  contact  mast  head  height. 

•  The  trainee  shall  know  his  height  of  eye. 

•  The  trainee  shall  know  how  to  determine:  size  and  distance  relationships  to 
navigation  aids,  contact  length,  distance  to  the  horizon,  hull  down,  and  angle  on  the 
bow. 

3C.  Understanding 
Advance  and 

Transfer 

•  The  trainee  shall  understand  the  concepts  of  advance  and  transfer. 

•  The  trainee  shall  understand  ship  characteristics  like  tactical  diameter  of  own  ship. 

•  The  trainee  shall  understand  the  criticality  of  turning  the  vessel  the  wrong  way. 

•  The  trainee  shall  imderstand  the  principles  of  conning  the  ship  through  turns. 

•  The  trainee  shall  understand  when  to  turn  the  ship  based  on  the  use  of  a  slide  bar. 

•  The  trainee  shall  understand  the  principles  of  compensation. 

•  The  trainee  shall  compensate  for  set  and  drift  when  making  turns, 

•  The  trainee  shall  check  that  the  next  channel  is  clear  prior  to  turning. 

•  The  trainee  shall  not  drive  based  solely  on  the  Navigator’s  recommendations. 

4C.  Understanding  the 
Effects  of  Tides, 
Currents,  Wind,  and 
Seas 

•  The  trainee  shall  understand  how  the  wind  affects  the  height  of  seas. 

•  The  trainee  shall  imderstand  that  current  and  tides  tend  in  the  direction  of  the 
natural  geography. 

•  The  trainee  shall  understand  the  relationship  of  the  estimated  winds  associated  with 
various  sea  heights. 

•  The  trainee  shall  understand  that  sea  height  influenced  by  wind  speed  can  give 
false  indications  of  the  actual  direction  of  currents. 

5C.  Understanding  Rules 
of  the  Road 

•  The  trainee  shall  comprehend  the  criticality  of  Rules  of  the  Road. 

•  The  trainee  shall  correctly  exercise  Rules  of  the  Road  by  taking  appropriate 
actions  in:  overtaking,  meeting,  passing,  and  crossing  situations. 

•  The  trainee  shall  understand  the  rules  for  sound  signals  and  responses. 

•  The  trainee  shall  take  appropriate  action  when  nearing  a  bend  in  the  channel. 

•  The  trainee  shall  take  appropriate  actions  to  avoid  collisions. 

6C.  Understanding 

Relative  Motion 
(Direction  &  Speed) 

•  The  trainee  shall  understand  true  and  relative  bearing  and  their  significance. 

•  The  trainee  shall  be  able  to  convert  relative  to  true  and  true  to  relative. 

•  The  trainee  shall  be  able  to  determine  the  relative  direction  of  contacts. 

•  The  trainee  shall  be  able  to  determine  own  ship’s  motion  relative  to  fixed  objects. 
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APPENDIX  B:  VESUB  Training  Objectives  (Continued) 


COGNITIVE 

COMPONENTS 

TRAINING  OBJECTIVES 

7C.  Understanding 

Methods  to 

Differentiate  and 
Prioritize  Traffic 

Contacts 

•  The  trainee  shall  be  able  to  classify,  differentiate,  and  prioritize  various  types  of 
.  contacts  and  other  obstacles. 

•  The  trainee  shall  understand  safe  distances  to  hazards. 

•  The  trainee  shall  be  able  to  effectively  determine  contacts  of  interest. 

•  The  trainee  shall  correctly  assign  master  control  numbers  to  contacts  of  concern. 

•  The  trainee  shall  maintain  awareness  of  contacts  in  relation  to  own  ship. 

•  The  trainee  shall  prompt  personnel  for  supporting  information. 

•  The  trainee  shall  drop  contacts  of  interest  when  no  longer  of  concern. 

•  The  trainee  shall  be  able  to  correctly  determine  contact’s  angle  on  the  bow. 

8C.  Understanding  Ship’s 
Operation  Under 

Harbor  Directives 

•  The  trainee  shall  understand  harbor,  port  limitations,  restrictions,  &  regulations. 

9C.  Understanding 

Methods  to  Deal  with 
Uncooperative  Traffic 

•  The  trainee  shall  take  proper  and  effective  actions  to  avoid  encounters  with 
uncooperative  traffic. 

IOC.  Understanding 

Correct  Operation  of 
Ship’s  Systems 

•  The  trainee  shall  understand  the  correct  operation  of  bridge  equipment. 

•  The  trainee  shall  verify  rudder  orders  by:  visually  checking  the  rudder  and  the 
bridge  suitcase  indicator. 

•  The  trainee  shall  verify  engine  orders  by:  checking  the  bridge  suitcase  indicator 
and  observing  screw  wash. 

lie.  Understanding  When 
and  How  to  Take 
Corrective  Actions 

•  The  trainee  shall  understand  emergency  operating  procedures. 

12C.  Understanding 

Effective 

Communication 

Procedures 

•  The  trainee  shall  speak  clearly. 

•  The  trainee  shall  use  correct  terminology. 

•  The  trainee  shall  effectively  communicate  with  each  station  using  required 
terminology. 

•  The  trainee  shall  acknowledge  all  reports  and  repeat  backs. 

•  The  trainee  shall  inform  appropriate  personnel  about  his  actions. 

•  The  trainee  shall  not  clutter  the  circuits. 
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APPENDIX  C 

Demographic  Information  on  Trainees  in  the  VESUB  TEE 


1  Ss# 

Rank 

Age 

Assignment 

Exp 

Glas 

ConLen 

Astig 

Vision 

ETC 

35 

Instmctor 

6 

Y 

Y 

N 

Good 

CIV 

45 

Harbor  Pilot 

18 

N 

N 

N 

Good 

ETCS 

33 

ANAV 

14 

N 

N 

N 

LT 

33 

9 

N 

N 

N 

LT 

26 

JO 

1 

Y 

N 

N 

Poor 

LT 

27 

Instractor 

4 

N 

N 

N 

Medium 

LTJG 

25 

JO 

0.5 

Y 

N 

N 

Poor 

LT 

27 

Instractor 

3 

N 

Y 

N 

LT 

26 

JO 

2 

Y 

N 

N 

Medium  | 

LT 

31 

5 

Y 

N 

Y 

ENS 

23 

JO 

0 

Y 

N 

N 

Good 

LTJG 

25 

RCA 

0 

N 

N 

N 

Good 

LT 

33 

Instructor 

4 

Y 

N 

Y 

Good 

LTJG 

JO 

1 

Y 

N 

N 

Medium 

LTJG 

29 

JO 

1 

N 

N 

N 

Good 

LCDR 

36 

xo 

12 

Y 

N 

N 

Medium 

LT 

27 

Asst  Ops 

3 

Y 

N 

Y 

Poor 

LT 

29 

JO 

3 

N 

Y 

N 

Poor 

LTJG 

25 

JO 

0.5 

Y 

N 

N 

Poor 

CDR 

38 

SUBLANT 

Staff 

16 

N 

N 

Very  Good 

Notes:  N  =  Norfolk;  G  =  Groton 

Assignment  =  Current  Assignment;  Exp  =  Yrs.  of  Ship  Handling  Experience; 
Glas  =  Wear  Glasses?  ConLens  =  Wear  Contact  Lenses?  Astig  =  Astigmatism? 
Vision  =  Self  Rating  on  Uncorrected  Vision 
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APPENDIX  C: 

Demographic  Information  on  Trainees  in  the  VESUB  TEE  (Continued) 


Ss# 

Rank 

Age 

Exp 

Glas 

ConLen 

Astig 

Vision 

G-1 

LT 

33 

Instructor 

3 

N 

N 

N 

Good 

G-2 

LTJG 

24 

.25 

Y 

N 

N 

Good 

G-3 

LTJG 

24 

JO  Billfish 

.25 

Y 

N 

Y 

Medium 

G-4 

LTJG 

26 

JO  Connecticut 

1 

N 

N 

N 

Good 

G-5 

LTJG 

24 

JO 

Philadelphia 

0 

N 

N 

Y 

Good 

G-6 

LTJG 

24 

JO  Coimecticut 

.5 

N 

N 

N 

Good 

G-7 

ENS 

25 

JO  Boston 

.3 

Y 

Y 

N 

G-8 

ENS 

32 

JO  Miami 

.5 

N 

N 

N 

Good 

G-9 

LT 

28 

SSCOL  Staff 

3 

Y 

Y 

N 

Poor 

G-10 

LT 

29 

Instructor 

3 

N 

Y 

Y 

Poor 

G-11 

ENS 

26 

SOBC 

0 

Y 

Y 

N 

Medium 

G-12 

LT 

29 

Instructor 

3 

Y 

N 

N 

Medium 

G-13 

ENS 

23 

SOBC 

0 

N 

Y 

Y 

G-14 

ENS 

30 

SOBC 

0 

Y 

Y 

Y 

Good 

G-15 

ENS 

23 

SOBC 

0 

N 

N 

N 

Good 

G-16 

LT 

27 

Instructor 

3 

N 

Y 

Y 

Poor 

G-17 

ENS 

25 

SOBC 

0 

Y 

Y 

N 

Med/Poor 

G-18 

LTJG 

30 

JO  Miami 

1.5 

N 

N 

N 

G-19 

LTJG 

23 

SOBC 

0 

Y 

N 

N 

Poor 

G-20 

LTJG 

25 

JO  Miami 

1 

Y 

Y 

N 

Poor 

G-21 

LT 

27 

Communicator 
,  Augusta 

3 

N 

N 

N 

Medium' 

G-22 

LT 

30 

SUBSCOL 

Staff 

4 

N 

N 

Y 

Good 

Notes:  N  =  Norfolk;  G  =  Groton 

Assignment  =  Current  Assignment  (JO  =  Junior  Officer;  SSCOL  =  Submarine  School; 
SOBC  =  Submarine  Officer  Basic  Course);  Exp  =  Yrs.  of  Ship  Handling  Experience; 
Glas  =  Wear  Glasses?  ConLens  =  Wear  Contact  Lenses?  Astig  =  Astigmatism? 
Vision  =  Self  Rating  on  Uncorrected  Vision 
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APPENDIX  D 

Demographic  Information  on  Observers  of  the  VESUB  TEE 


Rank 

E!3i 

Exp 

Location 

Civ 

55 

Chief  Harbor  Pilot 

20 

Norfolk 

LT 

29 

Instructor - 
SUBTRAFAC 

4 

Norfolk 

LCDR 

- 

Exec.  Officer  - 
USS  Hampton 

Norfolk 

COMSUBRON 

Four 

23 

Groton 

LT 

28 

3 

Groton 

CDR 

41 

Dir  Officer 
Training 
NAVSUBSCOL 

23 

Groton 

LCDR 

34 

NAVSUBSCOL - 
N26 

10 

Groton 

LT 

29 

Instructor, 

NAVSUBSCOL 

5 

Groton 

LT 

31 

SUBRONFour 

6 

Groton 

LT 

26 

NAVSUBSCOL - 
COMS  COR 

1 

Groton 

STSCS 

41 

NAVSUBSCOL - 
Dept  Master  Chief 

23 

Groton 

ENS 

23 

SOBC 

0 

Groton 

Notes:  Assignment  =  Current  Assignment;  Exp  =  Yrs.  of  Ship  Handling  Experience; 
COMSUBRON  =  Commander,  Submarine  Squadron;  SOBC  =  Submarine 
Officer  Basic  Course;  NAVSUBSCOL  =  Naval  Submarine  School 
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APPENDIX  E 

VESUB  TEE  Trainee  Demographic  Data  Collection  Form 


VESUB  TRAINING  EFFECTIVENESS  EVALUATION 


Background  Information: 


NAME  &  RANK  _ ^ _ 

AGE  _ 

CURRENT  ASSIGNMENT  _ 

YEARS  OF  SHIP  HANDLING  EXPERIENCE  _ 

Do  you  wear  glasses?  Y  /  N 

Do  you  wear  contact  lenses?  Y  /  N 

Do  you  have  an  astigmatism?  Y  /  N 

Please  rate  your  uncorrected  vision. 

Very  Good  Good  Medium  Poor  Very  Poor 


A 
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APPENDIX  F 

VESUB  TEE  “COMFORT”  QUESTIONNAIRE 

Pre-Experience  Comfort  Questionnaire 


Instructions :  Please 

circle  the 

severity  of 

any  symptoms 

that  apply  to  you 

riaht  now. 

(0) 

(1) 

(2) 

(3) 

1. 

General  Discomfort 

None 

Slight 

Moderate 

Severe 

2. 

Fatigue 

None 

Slight 

Moderate 

Severe 

3. 

Headache 

None 

Slight 

Moderate 

Severe 

4. 

Eye  Strain 

None 

Slight 

Moderate 

Severe 

5. 

Difficulty  Focusing 

None 

Slight 

Moderate 

Severe 

6. 

Increased  Salivation 

None 

Slight 

Moderate 

Severe 

7. 

Sweating 

None 

Slight 

Moderate 

Severe 

8. 

Nausea 

None 

Slight 

Moderate 

Severe 

9. 

Difficulty 

Concentrating 

None 

Slight 

Moderate 

Severe 

10 

.  Fullness  of  Head* 

None 

Slight 

Moderate 

Severe 

internal  pressure  in  head,  similar  to  sinus  pressure 

11 

.  Blurred  Vision 

None 

Slight 

Moderate 

Severe 

12 

.  Dizzy  (Eyes  Open) 

None 

Slight 

Moderate 

Severe 

13 

.  Dizzy  (Eyes  Closed) 

None 

Slight 

Moderate 

Severe 

14 

.  Vertigo** 

None 

Slight 

Moderate 

Severe 

**Vertigo  is  a  disordered 

state  in  which  the  person  or  his/her  surroundings 

seem  to  whirl  dizzily;  giddiness 

15 

.  Stomach  Awareness  *  *  * 

None 

Slight 

Moderate 

Severe 

***Stomach  awareness  is  usually  used 

to  indicate 

a  feeling  of 

discomfort  which 

is 

just  short  of  nausea 

16 

.  Burping 

None 

Slight 

Moderate 

Severe 

Are  there  any  other  symptoms  that  you  are  experiencing  right-  now?  If  so, 
please  describe  the  symptom (s)  and  rate  their  severity  on  the  other  side. 
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APPENDIX  G 

VESUB  TEE  TRAINEE  OPINION  DATA  COLLECTION  FORM 

PLEASE  ANSWER  THE  FOLLOWING  QUESTIONS  ABOUT  THE  VESUB  SYSTEM 
(please  circle  the  response  that  most  closely  matches  your  assessment) 


1.  I  had  a  positive  learning  experience  in  the  VESUB  system, 

strongly  agree  agree  neutral  disagree  strongly  disagree 

2.  Training  with  VESUB  will  increase  my  confidence  in  my  ship  handling 
skills . 

strongly  agree  agree  neutral  disagree  strongly  disagree 

3.  I  was  able  to  fully  adjust  the  VESUB  HMD  to  meet  my  specific  visual  needs, 

strongly  agree  agree  neutral  disagree  strongly  disagree 

4.  I  was  able  to  fully  adjust  the  VESUB  HMD  to  comfortably  fit  my  head, 

strongly  agree  agree  neutral  disagree  strongly  disagree 

5.  VESUB  should  be  used  for  introductory  ship  handling  training, 

strongly  agree  agree  neutral  disagree  strongly  disagree 

6.  VESUB  should  be  used  for  refresher  training  for  ship  handling  skills, 

strongly  agree  agree  neutral  disagree  strongly  disagree 

7.  VESUB  should  only  be  used  in  a  classroom  (shore-based  training) . 

strongly  agree  agree  neutral  disagree  strongly  disagree 

8.  VESUB  should  be  installed  on  submarines  for  mission  rehearsal  and 

refresher  training.  . .  - 

strongly  agree  agree  neutral  disagree  strongly  disagree 

9.  Please  provide  any  additional  suggestions  that  would  improve  the  realism, 
performance  or  training  effectiveness  of  VESUB.  (Please  continue  on  the  back 
if  necessary.) 
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APPENDIX  H 

VESUB  TEE  Observer  Questionnaire 

VESUB  TRAINING  EFFECTIVENESS  EVALUATION 
Observer  Questionnaire 


Background  Information: 

NAME  &  RANK  _ 

AGE  _ 

CURRENT  ASSIGNMENT  _ 

YEARS  OF  SHIP  HANDLING  EXPERIENCE 


PLEASE  ANSWER  THE  FOLLOWING  QUESTIONS  ABOUT  THE  VESUB  SYSTEM 
(please  circle  the  response  that  most  closely  matches  your  assessment) 


1.  The  VESUB  system  provides  a  positive  learning  experience, 

strongly  agree  agree  neutral  disagree  strongly  disagree 

2.  Training  with  VESUB  will  increase  a  trainee's  confidence  in  his  ship 
handling  skills. 

agree  agree  neutral  disagree  strongly  disagree 

should  be  used  for  introductory  ship  handling  training, 
agree  agree  neutral  disagree  strongly  disagree 

should  be  used  for  refresher  training  for  ship  handling  skills, 
agree  agree  neutral  disagree  strongly  disagree 

should  only  be  used  in  classroom  (shore-based  training) . 
agree  agree  neutral  disagree  strongly  disagree 

should  be  installed  on  submarines  for  mission  rehearsal  and 
training. 

agree  agree  neutral  disagree  strongly  disagree 

7 .  Please  provide  any  additional  suggestions  that  would  improve  the  realism, 
performance  or  training  effectiveness  of  VESUB.  (Please  continue  on  the  back 
if  necessary.) 


strongly 

3 .  VESUB 
strongly 

4.  VESUB 
strongly 

5.  VESUB 
strongly 

6.  VESUB 
refresher 

strongly 
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APPENDIX  I 

VESUB  TEE  SCENARIO  SCRIPT  (INBOUND  FIRST) 


PREBRIEF 

VESUB;  Virtual  Environment  for  Submarine  OOD  Ship  Handling  Training.  VESUB  is  a 
Research  &  Development  Technology  Demonstration  Project.  There  are  several  known  flaws  in 
the  system.  We  are  aware  of  these  and  will  correct  them  in  the  future.  WeTl  point  these  out  to 
you  and  coach  you  through  them  while  you  are  in  the  system. 

History:  This  is  the  fourth  and  final  year  of  the  VESUB  project.  We  began  with  the 
development  of  a  feasibility  demonstration  system  that  was  very  simple,  but  allowed  us  to  obtain 
inputs  fi-om  fleet  experts  on  the  features  that  were  required  in  VESUB.  It  took  about  a  year  to 
collect  and  organize  all  the  inputs  into  a  specification  for  the  system.  Two  years  of  development 
have  lead  to  the  system  you  will  experience  today. 

Purpose  and  organization  of  the  Training  Effectiveness  Evaluation  (TEEI:  The  purpose  of 
the  TEE  is  to  demonstrate  to  the  fleet  the  effectiveness  of  VE  training  technologies.  Also,  to 
collect  reactions  and  recommendations  on  how  to  improve  VESUB  for  operational  use  and  to 
improve  VE  technologies  in  general.  Also,  we’d  like  your  inputs  on  how  you  think  VESUB  can 
be  used  in  the  training  pipeline  and  how  VE  technologies  can  be  used  in  other  tasks. 

•  The  TEE  will  consist  of  three  scenarios.  The  first  scenario  will  allow  you  to  experience  and 
practice  all  system  capabilities.  The  second  scenario  will  train  you  in  several  harbor  ship 
handling  tasks  and  the  third  scenario  will  test  your  improvement  on  these  tasks. 

•  Because  your  experiences  might  influence  other  people’s  performance,  please  do  not  talk  to 
anyone  about  your  experiences  in  VESUB. 

•  Because  VE  technologies  are  new,  we  have  very  little  data  on  how  people  react  to  them.  We’d 
like  you  to  fill  out  a  series  of  “Comfort”  questioimaires  before  and  after  each  session  in  the 
VESUB.  If  AT  ANY  TIME  you  feel  uncomfortable,  tell  us  and  we  will  remove  you  firom  the 
system. 

Now,  please  take  a  few  moments  and  fill  out  the  background  information  and  the  first  “comfort” 
questiormaire. 

Do  you  have  any  questions? 
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APPENDIX  I:  VESUB  TEE  SCENARIO  SCRIPT  (INBOUND  FIRST) 

(Continued) 

VESUB  TEE  Scenario  Events 
Scenario  1  -  Orientation  (Norfolk)  (Page  1  of  3) 


Event 

Instructor  Script 

1.  Pre-brief: 

a.  Restate 
organization  of 
scenarios. 

b.  Show  trainee  the 
Norfolk  Chart  anti 
give  basic  overview  of 
harbor. 

•  The  TEE  will  consist  of  3  scenarios.  The  first  scenario  will  allow  you  to 
experience  and  practice  all  system  capabilities.  The  second  scenario  will  train  you 
in  several  harbor  ship  handling  tasks  and  the  third  scenario  will  test  your 
improvement  on  these  tasks. 

•  VESUB  has  modeled  two  databases:  Norfolk,  VA  and  Kings  Bay,  GA.  This 
orientation  scenario  will  take  place  in  Norfolk.  The  training  and  testing  scenarios 
will  take  place  in  Kings  Bay. 

•  Here  is  the  Norfolk  chart.  You  will  be  located  outside  pier  10  and  will  be  headed 
outbound.  While  you  are  in  the  virtual  world,  you  will  be  able  to  call  up  the  chart  at 
any  time  by  saying  the  word  “chart.” 

2.  Trainee  is  shown  all 
equipment: 

•  Bridge  Mock- 
up 

•  Microphone 

•  HMD 

The  Bridge  Mock-UP  is  intended  to  provide  spatial  orientation  and  a  safe 
environment  while  you  are  in  the  virtual  world.  Fell  free  to  grab  the  mock-up  to 
stabilize  &  orient  yourself  We’ve  found  that  side  effects  from  VE  can  be  greatly 
reduced  by  touching  something  in  the  real  world.  Also,  feel  free  to  move  arouncl  the 
bridge  to  obtain  better  views.  The  system  senses  where  you  are  looking  and  repaints 
the  scene. 

•You  will  be  issuing  Helm  orders  through  this  push-to-talk  microphone.  You  will 
also  be  able  to  request  info  from  the  Navigator.  You  must  use  station  IDs  &  correct 
phrasing.  You  also  need  to  acknowledge  repeat  backs  and  reports.  Just  speak  in  a 
normal  tone  with  the  mic  about  2-3  inches  from  your  mouth.  If  the  system  doesn’t 
understand  you,  it  will  reply  Say  again,  Sir.  The  instructor  may  intervene  &  tell  you 
to  ignore  the  say  again  if  the  system  is  causing  the  problem.  Try  not  to  hesitate  in 
the  middle  of  a  command.  If  you  wish  to  issue  commands  to  other  stations  or 
request  information  from  them  (e.g.,  Control,  Contact  Coordinator)  please  do  not 
use  the  mic.  Just  speak  directly  to  the  instr  who  will  play  these  roles.  You  may 
occasionally  hear  an  erroneous  report  or  repeat  back  (e.g.  “on  outboard”).  Please 
ignore  these  reports.  At  ANY  time,  please  talk  to  us  about  the  system  or  scenario 

(e.g.  my  CO  would _ ).  You  may  call  up  a  chart,  course  card,  or  binocs  by  saying 

those  words  into  the  mic. 

•This  is  a  high  resolution  head-mounted  display  or  HMD.  You  focus  each  eye 
independently  with  the  slide  bars  on  the  right  and  left  of  the  underside.  There  are  2 
fit  adjustments:  tightness  around  your  head  and  height  of  the  eyepieces  -  Try  various 
positions  for  best  fit.  Try  to  keep  the  HMD  level  for  best  fit. 

3.  Trainee  is  placed  in  HMD  —  Go  to  the  Next  Page 
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APPENDIX  I;  VESUB  TEE  SCENARIO  SCRIPT  (INBOUND  FIRST) 

(Continued) 

VESUB  TEE  Scenario  Events 
Scenario  1  -  Orientation  (Norfolk)  (Continued)  (Page  2  of  3) 


Event 

Instructor  Script 

4.  Trainee  explores 
virtual  world 

•  Look  around  the  virtual  world.  You  have  a  complete  360  degree  environment. 
Notice  you  can  see  the  rudder  astern,  although  you  may  need  to  look  around 
obstacles  or  hang  over  the  side  to  see  it. 

5.  Go  to  UNFREEZE 

•  Show  Chart 

•  Show  Course  Card 

•  Show  Binoculars 

•  Look  to  the  Starboard  side  and  say  the  word  “Chart,”  This  chart  will  dynamically 
change  as  you  move  through  the  harbor.  You  may  toggle  the  chart  on  &  off  by 
saying  “Chart.”  Please  try  now.  Find  the  optimal  distance  to  read  the  chart.  If  you 
are  too  close  you  will  poke  your  head  right  through  it. 

•  Look  to  the  Port  side  and  say  the  word  “Course  card.”  You  may  “toggle”  the 
course  card  on  and  off  at  any  time  by  saying  “course  card.”  Please  try  now. 

•  Look  into  the  distance  and  say  the  word  “binoculars.”  You  now  have  a  7.5  power 
magnified  view.  This  is  the  same  power  as  the  binoculars  on  the  submarine  bridge. 
You  can  “toggle”  the  binoculars  on  and  off  at  any  time  by  saying  the  word 
“binoculars”  into  the  mic.  Grab  the  HMD  as  you  would  real  binocs  to  steady  the 
view.  Please  toggle  the  binoculars  on  and  off  a  couple  of  times. 

6.  Show  Contacts  at 
Various  Ranges 

Show  how  the 
contacts  look  through 
binoculars 

We’d  like  to  demonstrate  how  various  contacts  appear  at  different  distances 

•  (#13)  Look  at  the  contact  at  180R,  185T.  It  is  a  tug  with  sand  barge  at  a  range  of 
1600yds.  It  has  a  S45  AOB.  Now  turn  on  the  binoculars 

•  (#6)  Look  at  the  contact  at  237R,  242T.  It  is  a  sailboat  at  a  range  of 2500yds,  with 
aS30AOB.  Now  turn  on  the  binocs. 

•  (#4)  281R,  286T:  Cabin  Cruiser,  2100yds,  P75  AOB 

•  (#15)  013R,  018T:  Fishing  Trawler,  2100yds,  P135  AOB 

•  (#17)348R,  363T:  Black  Container  Ship,  4800yds,  S90  AOB 

•  (# 1 8)  308R,  3 1 3T:  Blue  Container  Ship,  4600yds,  P23  AOB 

7.  Show  different 
visibility  conditions 
and  time  of  day 

We’d  like  to  show  you  how  the  system  represents  different  levels  of  visibility.  You 
are  now  in  unlimited  vis.  We’ll  make  it  foggy  with  2nm  visibility.  Now, 

'  lmiL....5iim 

•  Here’s  how  it  looks  with  rain  approaching  &  lOnm  vis. 

•  You  are  now  at  1030.  Here’s  what  different  times  of  day  look  like  (0630,  1200, 
2300) 

8.  Ask  Trainee  to  look 
at  navigation  aids  and 
find  them  on  the  chart 

We’d  like  you  to  find  several  navigation  aids  &  then  locate  them  on  the  chart. 

•  First,  look  at  the  buoy  #2  located  at  342R.  Try  the  binoculars.  Now,  try  to  find  it 
on  the  chart. 

•  Look  at  Day  beacon  ART  #8  located  at  21 IR.  binoc  chart 

•  Please  look  at  Sewell’s  Pt.  Tower  located  at  043  true.  Try  binoculars.  Now  try 
to  find  it  on  the  Chart 

Go  to  the  Next  Page 
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APPENDIX  I:  VESUB  TEE  SCENARIO  SCRIPT  (INBOUND  FIRST) 

(Continued) 

VESUB  TEE  Scenario  Events 

Scenario  1  -  Orientation  (Norfolk)  (Continued)  (Page  3  of  3) 


Event 

Instructor  Script 

9.  Ask  trainee  to  issue 
speed  orders 

•  Try  several  speed  commands.  Remember  to  use  station  identifiers  and  to 
acknowledee  the  reports. 

•  Ahd  1/3;  All  Back  emergency;  All  stop;  Ahd  2/3 

10.  Ask  trainee  to 
issue  several  rudder 
commands 

•  Try  several  rudder  commands.  Remember  to  use  station  identifiers  and  to 
acknowledge  the  reports. 

•  Large  rudder  orders  will  turn  you  quickly  and  slow  your  headway  -  small  mdder 
orders  will  have  only  limited  effect. 

•  The  standard  rudder  for  executing  turns  in  our  navplan  is  15  degrees  right  or  left. 

•  Try  some  rudder  commands  now.  Remember  to  visually  site  the  rudder  position 
astern  as  well  as  the  indicator  on  the  bridge  suitcase. 

•  Right  15  degs  rudder;  Shift  your  rudder;  Steady  course  004 

11.  Show  Channel 
Boundaries  and 
Centerline 

•  Here  are  two  instructional  tools  to  help  the  trainee  understand  his  position  in  the 
channel.  We  can  highlight  the  channel  boundaries,  like  this.  We  can  also  highlight 
the  centerline,  like  this.  They  can  be  removed,  like  this. 

12.  Demonstrate  Nav 
reports 

•  You  will  be  able  to  receive  information  from  the  Navigator,  either  produced  by  the 
system  or  the  instructor.  You  can  also  request  information  from  the  Nav  with  the 
microphone. 

•  Here’s  some  examples  of  the  information  the  system  can  provide,  (Distance  off 
track;  Course  to  regain  track; 

Distance  to  next  turn;  Next  course) 

•  Now,  try  to  ask  the  Navigator  for  “distance  to  NEXT  turn” 

13.  Have  trainee  make 
a  turn  after  Nav  marks 
it 

Please  get  underway  at  ahead  2/3.  When  you  hear  the  navigator  mark  the  turn,  give 
the  appropriate  helm  order  and  next  course. 

14.  Ask  if  he  has  any 
questions 

Do  you  have  any  questions  about  the  system? 

Orientation  Scenario 
Ends 

Please  fill  out  the  second  comfort  questionnaire 
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APPENDIX  I;  VESUB  TEE  SCENARIO  SCRIPT  (INBOUND  FIRST) 

(Continued) 

VESUB  TEE  Scenario  Events 
Scenario  2  -  Training  (Kings  Bay  Inbound) 

Prebrief 

In  this  scenario  you  will  be  in  a  726  Trident  on  an  inbound  course  into  Kings  Bay,  GA.  Let’s 
take  a  look  at  the  chart.  You  enter  from  the  Southeast  (not  shown  on  this  chart)  on  Trident 
Range.  Then  you  turn  west  onto  Entrance  Channel,  which  is  protected  by  jetties  and  breakwaters 
that  will  limit  wave  height  and  cross  currents.  Set  &  drift  on  the  Entrance  Channel  will  typically 
be  east  and  west  based  on  tides  &  currents.  Some  of  the  major  geographic  features  of  the  harbor 
include:  Cumberland  Island  Sound,  ST  Mary’s  and  Amelia  Rivers  all  of  which  will  cause 
currents  that  will  affect  your  ship,  Cumberland  Island  (served  by  a  Ferry  from  ST  Mary’s),  and 
Amelia  Island,  with  FT  Clinch  on  its  northern  shore.  All  of  the  channel  legs  of  this  harbor  are 
supported  by  range  markers,  either  astern,  forward,  or  both.  For  example.  Range  “A”  has  range 
markers  both  astern  and  forward.  Other  traffic  in  the  harbor  may  include:  coastal  merchants 
going  to  the  paper  mills  and  Amelia  Island,  fishermen,  pleasure  craft,  and  other  Navy  traffic. 

For  this  scenario: 

•  Visibility  is  unlimited 

•  You  are  at  Flood  Tide  and  currents  will  be  representative  of  those  in  Kings  Bay 

•  All  Navplan  course  changes  will  be  based  on  a  15  degree  rudder  and  a  speed  of  12  knots 

•  Total  time  for  the  scenario  is  about  35  minutes 

•  Red  and  yellow  soimdings  are  set  at  3  and  5  feet,  respectively 

•  You  may  encounter  typical  traffic  like  in  Kings  Bay 

•  Follow  standard  inland  rules  of  the  road. 

•  Additionally,  you  will  be  expected  to  abide  by  harbor  directives  provided  by  your  Navigator. 
For  example,  be  sure  to  slow  own  ship  when  passing  small  boats 

•  Permission  to  enter  port  has  already  been  obtained 

•  You  will  be  berthed  at  Trident  Refit  Wharf  #1 

Take  a  few  moments  to  review  the  chart  and  course  card.  You  will  begin  somewhere  on  the 
Entrance  range.  Remember,  you  will  be  able  to  call  up  the  chart  and  a  course  card  like  this  from 
the  bridge. 

•  When  we  ask  you  to  report  a  contact,  please  give  a  detailed  report,  including:  type  of  contact, 
range,  bearing,  AOB,  and  your  concern  about  the  contact. 

•  Any  Questions?  Let’s  go  to  the  bridge  mock-up. 
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APPENDIX  I:  VESUB  TEE  SCENARIO  SCRIPT  (INBOUND  FIRST) 

(Continued) 

VESUB  TEE  Scenario  Events 
Scenario  2  -  Training  (Kings  Bay  Inbound!  (page  1  of  8) 


Event 

Time 

Component 

Instructor  Script 

1.  Own  ship  is  placed 
on  Entrance  Range, 
2000yds  from  the  1st 
turn,  50yds  right  of 
track. 

While  in  Frozen 

Mode 

00:00 

IP,  Locating  &  ED 
Navigation  aids;  2P. 
Judging  Distance; 

2C.  Understanding 
relationship  of 
visual  cues  to 
chart(s) 

•  As  quickly  and  accurately  as  possible,  tell  us 
where  you  are  now  located? 

•  What  did  you  do  to  determine  your  location? 

•  What  is  your  distance  left  or  right  of  track?  (if 
not  included  above) 

•  How  did  you  determine  your  distance  L/R  or 
track?  (if  not  included  above) 

2.  Instructor 

Intervention 

00:00 

Your  Reported  Position  was: 

Your  Actual  Position  was: 

Finding  and  placing  own  ship  in  the  geographic 
field  is  an  event  that  the  OOD  will  experience  each 
time  that  he  relieves  the  watch.  His  first  task  is  to 
review  the  chart  and  become  familiar  with  the 
environment.  Next  he  surveys  the  visual  scene, 
looks  for  key  reference  points  and  objects, 
evaluates  these  and  correlates  them  to  his  chart  to 
confirm  ship’s  location. , 

(continue  on  next  page) 

irrfTr'wnvi'— 1 

1-6 


Technical  Report  98-003 


APPENDIX  I:  VESUB  TEE  SCENARIO  SCRIPT  aNBOUND  FIRST) 

(Continued) 

VESUB  TEE  Scenario  Events 
Scenario  2  -  Training  (Kings  Bay  Inboundi  (page  2  of  8) 


Event 

Time 

Component  |  Instructor  Script 

2.  Instructor 
Intervention 
(continued) 

00:00 

The  two  step  process  that  may  have  better  supported  your  efforts  in 
this  task  would  have  been  to  (1)  find  and  fix  the  position  of  at  least  two 
nav  aids.  One  near  the  beam  and  one  forward  of  your  position. 
Additionally,  use  of  closer  navigation  aids  like  buoys  and  channel 
markers  can  be  used  to  provide  you  with  placement  along  the  navigator’s 
track.  Next  (2)  it  is  necessary  to  sight  one  of  the  stationary  ranges  that  are 
placed  at  the  head  of  each  leg  of  this  harbor.  These  ranges  will  provide 
you  with  an  accurate  position  in  relation  to  the  center  of  the  channel.  The 
range  can  tell  you  when  you  are  right,  left,  or  center  in  the  channel. 

You  can  further  verify  positions  right  or  left  of  track  by  marking  your 
distance  between  buoys  as  you  pass  through  port  and  starboard  pairs, 
however,  these  are  not  always  positioned  in  tos  manner. 

Take  a  look  at  the  following  navigation  aids  in  the  visual  scene  (see  list 
that  he  didn’t  use  from  data  collection  form). 

3.  Trainee  is  asked  to 
report  all  visible 
contacts. 

00:00 

2P.  Judging 
distance 

4P.  ID  &  avoid 
obstacles 

2C.  Understanding 
Size/Range  & 

AOB 

7C.  Prioritize 
contacts 

Please  report  all  visible  contacts.  For  NOW  if  vou 
want  binocular  view,  ask  the  instructor.  This  is  also 
tme  for  “Chart.”  After  we  start  the  scenario,  you 
will  be  able  to  obtain  binocular  view  or  chart  by 
saying  “binoculars”  or  “chart”  into  the 
microphone. 

4.  Instructor 
Intervention 

During  this  event  you  were  asked  to  find  and  evaluate  all  contacts.  As  an 
OOD  you  will  need  to  be  cognizant  of  all  contact  traffic  &  maintain  a 
mental  picture  of  the  relationship  of  these  contacts  to  own  ship.  It  is 
important  that  you  find  and  evaluate  each  contact  for  bearing,  range, 
classification,  AOB,  and  concern  that  you  may  have  for  own  ship’s  safety 
based  on  their  position  in  the  harbor.  More  complex  tasks,  such  as 
determining  CPA  and  Rules  of  the  Road  all  begin  with  this  first  analysis. 

(go  to  next  page) 

. . . . . 

■■■ 
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APPENDIX  I:  VESUB  TEE  SCENARIO  SCRIPT  (INBOUND  FIRST) 

(Continued) 

VESUB  TEE  Scenario  Events 
Scenario  2  -  Training  (Kings  Bay  Inbound)  (page  3  of  8) 


Event 

Time 

4.  Instructor 

Intervention 

(continued) 

Of  our  contacts  you  found  of 

The  actual  contacts  were: 

(use  data  collection  form) 

Your  reported  contacts  were: 

(use  data  collection  form) 

5.  •  Give  trainee  the 
mic 

•  Go  to  RUN  Mode 

00:00 

•  When  we  go  to  run  mode,  the  first  thing  you 
hear  will  be  a  report  from  the  Navigator. 

•  Here  is  the  microphone. 

•  UNFREEZE  -  The  scenario  is  now  running 

6.  Report  from 

Navigator: 

00:02 

8P.  Detecting 
comms. 

•  Instructor  initiates  reports  from  the  Nav.  USE 
SYSTEM. 

•  distance  from  track 

•  recommended  course 

•  recommended  speed 

•  sounding 

•  set  &  drift 

•  distance  to  turn 

•  next  course 

7.  Trainee  orders  ship 
to  get  underway  and 
adjusts  course 

00:30 

IOC.  Correctly 
using  ship’s 
systems; 

12C.  Effective 
commimication 

Instructor  says,  “Recommend  you  Make  turns  for 
12  knots” 

8,  Man  Overboard 
(trainee  hears  report 
from  lookout  played  by 
instructor) 

02:30 

8P.  Detecting 
comms. 
lie.  Taking 
corrective  actions 
12C.  Effective 

comms. 

•  “Officer  of  the  Deck,  Man  Overboard  Port  Side” 

9.  Maneuvering 
(Instructor)  reports: 

02:40 

8P.  Detecting 
comms. 

•  “Bridge,  Maneuvering.  Shaft  is  stopped  and 
locked” 

Go  to  Next  Page 
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APPENDIX  I:  VESUB  TEE  SCENARIO  SCRIPT  (INBOUND  FIRST) 

(Continued) 


VESUB  TEE  Scenario  Events  (Continued) 

Scenario  2  -  Training  (Kings  Bay  Inboundi  (Continued  -  page  4  of  8) 


Event 

Time 

Instructor  Script 

10.  Report  from 

Control  (instructor)  - 
Tug  is  picking  up  man 
overboard 

04:00 

8P.  Detecting 
coimns. 

•“Bridge,  Navigator.  Have  contacted  C-tractor  to 
retrieve  man  overboard.  C-tractor  agrees  to  take 
man  overboard  retrieval  responsibilities. 
Recommend  returning  to  base  course  268  at  ahead 
2/3” 

11.  Trainee  gets  ship 
underway  again 

02:50 

lie.  Taking 
corrective  actions 
12C.  Effective 

comms. 

12.  Instructor 
Intervention 

Go  to 

The  most  important  action  that  the  OOD  may  take  on  the  surface  is  that  of  saving 
the  life  of  a  man  who  has  fallen  overboard.  The  ship’s  control  party  has  several 
immediate  actions  that  must  be  carried  out  in  this  casualty,  and  it  is  the 
responsibility  of  the  OOD  to  ensure  that  they  have  been  accomplished.  He  does 
this  through  issuing  backup  orders  and  through  verification  of  each  step. 

If  the  victim  has  survived  the  fall  into  the  channel  he  faces  an  immediate 
danger  from  the  ship’s  control  surfaces  and  the  screw.  The  OOD  takes  steps  to 
stop  the  shaft,  so  that  the  blades  of  the  screw  will  not  be  turning  should  they  make 
contact  with  the  man,  and  to  swing  the  ship  away  from  the  man. 

Turning  the  ship  is  accomplished  by  putting  a  full  rudder  on  in  the  direction  of 
the  man;  that  is,  if  he  has  fallen  off  of  the  starboard  side  the  ship  is  turned  with 
right  full  rudder  and  left  full  rudder  for  the  port  side. 

Your  actions  could  have  been  more  effective  if  you  had: 

1.  .  ,2.  .  ,3.  ,4. _ 

If  your  position  becomes  dangerous  and  you  need  to  put  on  a  backing  bell,  you 
need  to  sound  3  whistles  for  astern  propulsion. 

Additionally,  you  should  have  given  consideration  to  retrieval  of  the  victim. 
Using  your  bricige  to  bridge  radio  or  having  the  Navigator  communicate  with  the 
tug  to  arrange  for  a  personnel  pickup  would  have  allowed  you  to  continue  along 
your  track  while  taking  care  of  your  crew  member. 

■  III!  IIIIIII^M 
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APPENDIX  I:  VESUB  TEE  SCENARIO  SCRIPT  (INBOUND  FIRST) 

(Continued) 

VESUB  TEE  Scenario  Events  (Continned) 

Scenario  2  -  Training  (Kings  Bay  Inbound)  (Continued  -  page  5  of  8) 


Event 

Time 

Component 

Instructor  Script 

13.  Goto 

UNFREEZE 

Trainee  gets  ship 
underway  again 
(Trainee  may  try  to 
correct  ship’s  position. 
Should  as  a  minimum, 
evaluate  position  and 
ask  Nav  for  backup) 

04:15 

IOC.  Correct 
operation  of  ship’s 
systems 

12C.  Effective 

comms. 

•  (system)  “Bridge,  Navigator.  Recommend 
all  ahead  standard. 

•  (system)  “Bridge,  Navigator.  Recommend 
XXX  to  regain  track” 

•  (Instructor)  “B,  N.  Recommend  you  make 
turns  for  12  knots” 

14.  Trainee  commands 
turn  to  Range  A  after 
Nav  marks  turn 

Course  :  294 

07:30 

IOC.  Correct 
operation  of  ship’s 
systems 

12C.  Effective 

comms. 

15.  Nav  reports 
incorrect  position  and 
incorrect  course  to 
regain  track.  After 
steady  on  294.  (Left 
when  Right  or  Right 
when  Left) 

08:15 

lie.  When  &  how  to 
take  corrective 
actions 

12C.  Effective 

comms. 

Instmctor  says: 

•  If  Center  -  hold  you  left  of  track. 

Recommend  right  to  290  to  regain  track 

•  If  left  -  hold  you  right  of  track  -  recommend 
R290 

•  If  right  -  hold  you  left  of  track  -  recommend 
L298 

16.  Instructor 
Intervention 

We’ve  jus 
games,  bu 
other  peot 

t  given  you  an  incorrect  report  from  the  Navigator.  This  is  not  to  play 
t  to  point  out  that  the  OOD  is  in  charge  and  must  always  determine  if 
)le  have  made  a  mistake. 

17.  Trainee  commands 
turn  to  Range  A-1  after 
Nav  Marks  Turn 

Course:  302.5 

12:40 

12C.  Effective 

comms. 
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APPENDIX  I:  VESUB  TEE  SCENARIO  SCRIPT  aNBOUND  FIRST) 

(Continued) 

VESUB  TEE  Scenario  Events  (Continned) 

Scenario  2  -  Training  (Kings  Bay  Inbound¥Continned  -  page  6  of  8) 


Event 

Time 

Component 

Instructor  Script 

18.  Nav  reports  correct 
information  (short 
report)  After  steady  on 
302 

System  Generates: 

•  distance  to  turn 

•  next  course 

•  distance  from  track 

19.  Trainee  commands 
turn  to  Range  A-2  after 
Nav  Marks  Turn 

Course:  331.5 

20.  Navigator  short 
report  (after  steady  on 
331) 

System  Generates; 

•  distance  to  turn 

•  next  course 

•  distance  from  track 

21.  Trainee  commands 
turn  to  Range  B  after 
Nav  Marks  Turn 

Course;  350 

22.  Navigator  short 
report  (after  steady  on 
350) 

System  Generates: 

•  distance  to  turn 

•  next  course 

•  distance  from  track 

23.  Ferry  becomes 
visible  (possible). 

15:30 

4P.  id  &  avoid  obstacles 
7C.  Prioritize  contacts 
12C.  Effective  comms. 

24.  Trainee  commands 
turn  to  Ran^e  C  after 
Nav  Marks  Turn 

Course:  004 

17:30 

12C.  Effective  comms. 

25.  Navigator  FULL 
report-  (after steady 
on  004) 

System  Generates 

•  distance  from  track 

•  recommended  course 

•  recommended  speed 

•  sounding 

•  set  &  drift 

•  distance  to  turn 

•  next  course 
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APPENDIX  I:  VESUB  TEE  SCENARIO  SCRIPT  (INBOUND  FIRST) 

(Continued) 

VESUB  TEE  Scenario  Events  (Continued) 

Scenario  2  -  Training  (Kings  Bay  InboundV Continued  -  page  7  of  8) 


Event 

Time 

Component 

Instructor  Script 

Instructor:  “Bridge,  Contact 

Coordinator.  Hold  contact  at  xxx. 
Appears  to  be  a  ferry  boat.  Etc 

27.  OOD  maneuvers 
own  ship  to  right  for  P 
to  P  passage.  Instructor 
controls  Ferry  for  Port- 
to-port  passage 

20:00 

28.  Trainee  increases 
speed 

21:00 

29.  Go  to  FREEZE 
Instructor 

Intervention 

During  crossing  situations  like  you  just  encountered  the  OOD 
should:  1)  Evaluate  the  relative  motion  of  contacts  to  determine  if  of 
concern,  or  will  affect  own  ship’s  progress  along  the  track  You 
.  2)  Communicate  with  other  vessels  to  determine  their 
intended  actions.  You  .3)  Slow  the  ship’s  headway  when  he 

will  hazard  another  vessel  You  .4)  Maneuver  to  split  the 

channel  when  necessary.  You  .5)  Report  to  the  Commanding 

Officer  all  contacts  of  concern  per  the  CO’s  standing  orders. 

These  actions  will  often  be  done  simultaneously  and  are  subject  to 
the  OOD’s  understanding  of  ship’s  position  and  contact  actions.  One 
often  overlooked  resource  in  contact  management  is  the  chart.  The 
OOD  should  evaluate  the  contacts  that  he  holds  and  refer  to  the  chart 
to  determine  what  constraints  they  are  subject  to  as  they  move  along 
the  channel.  This  will  help  him  to  sort  out  their  impact  on  his 
transit. 

30.  Yellow  soimding 

23:30 

“Bridge,  Navigator.  Yellow  sounding, 

4ft” 

31.  Trainee  commands 
to  regain  speed  and 
course 

Go  to  the  Next  Page  I 
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APPENDIX  I:  VESUB  TEE  SCENARIO  SCRIPT  (INBOUND  FIRST) 

(Continued) 

VESUB  TEE  Scenario  Events  (Continued) 

Scenario  2  -  Training  (Kings  Bay  InboundlfContinued  -  page  8  of  8) 


Event 

Time 

Component 

Instructor  Script 

32.  Instructor  (Nav) 
gives  info,  if  trainee 
does  not  request  within 
30  secs. 

24:00 

33.  Go  to  FREEZE. 

Instructor 

Intervention 

In  the  case  of  a  sounding  that  does  not  match  with  charted  water  depth  the  nav 
party  will  try  to  confirm  the  sounding  &  fix  the  position  of  the  ship. 

A  yellow  sounding  is  a  trip  point,  set  by  the  navigator  and  approved  by  the  CO, 
that  indicates  that  the  ship  may  be  standing  into  danger.  It  requires  OOD  to  act 
aggressively  to  place  ship  on  safest  course  &  heading  into  safe  water. 

To  review  the  required  actions,  the  OOD  should: 

•  Slow  the  ship's  headway.  You:  Did  /  Didn’t 

•  Confirm  the  direction  to  good  or  best  water.  You:  Did  /  Didn’t 

•  Maneuver  to  seek  center  channel  or  best  water  when  possible.  You:  Did  / 

Didn’t 

•  Fix  the  ship’s  position  (Nav  team  will  do  automatically).  You:  Did  /  Didn’t 

•  Sight  the  range  ahead  or  astern  as  available.  You:  Did  /  Didn’t 

•  Report  to  the  CAPT  the  status  of  the  sounding  and  his  actions  to  properly  and 
safely  position  the  ship.  You:  Did /Didn’t 

These  actions  will  often  be  done  simultaneously  &  are  subject  to  the  OOD’s 
understanding  of  ship’s  position  when  the  yellow  sounding  was  gained. 

34,  Goto 

UNFREEZE. 

Trainee  increases 
speed  &  sets  new 
course. 

24:30 

35.  Scenario  Ends 
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APPENDIX  I:  VESUB  TEE  SCENARIO  SCRIPT  (INBOUND  FIRST) 

(Continued) 

VESUB  TEE  Scenario  Events 
Scenario  3  -  Testing  (Kings  Bay  Outbound) 

Prebrief 

In  this  scenario  you  will  be  in  a  726  Trident  on  an  outbound  course  from  Kings  Bay.  Let’s  take  a 
look  at  the  chart.  Kings  Bay  Naval  facility  is  protected  to  sea  by  Crab  Island,  a  man-made  island 
from  the  dredging  of  the  harbor.  As  you  proceed  outbound,  you  travel  through  Cumberland 
Sound.  You  intersect  St.  Mary’s  and  Amelia  Rivers,  which  produce  currents  that  will  affect  your 
ship  due  to  set  &  drift.  The  port  includes  maintenance  piers.  Explosive  Handling  Wharves 
(large,  yellowish  buildings),  and  a  dry  dock.  Warrior  Reach  area  includes  a  Magnetic  Silencing 
Facility  (MSF)  and  degaussing  range  and  may  have  large  vessels  including  other  warships.  All 
of  the  legs  of  the  channel  include  range  markers  either  astern,  forward,  or  both.  For  example. 
Range  “D”  has  ranges  astern.  Other  traffic  in  the  harbor  may  include:  coastal  merchants  going 
to  the  paper  mills  and  Amelia  Island,  fishermen,  pleasure  craft,  and  other  Navy  traffic. 

For  this  scenario: 

•  Visibility  is  unlimited 

•  You  are  at  Ebb  Tide  and  currents  will  be  representative  of  those  in  Kings  Bay 

•  All  Navplan  course  changes  will  be  based  on  15  degree  rudder  and  a  speed  of  12  knots 

•  Total  time  for  the  scenario  is  about  35  minutes 

•  Red  and  yellow  soxmdings  are  set  at  3  and  5  feet,  respectively 

•  You  may  encounter  typical  traffic  that  could  be  seen  in  Kings  Bay 

•  Follow  standard  inland  rules  of  the  road. 

•  Additionally,  you  will  be  expected  to  abide  by  harbor  directives  provided  by  your  Navigator. 
For  example,  be  sure  to  slow  own  ship  when  passing  small  boats 

Now,  take  a  few  moments  to  review  the  chart  and  course  card.  Some  of  the  major  geographic 
features  include:  the  base  and  the  facilities  already  mentioned,  Cumberland  Island  (which  is 
served  by  a  Ferry  from  ST  Mary’s),  and  Amelia  Island,  with  Ft  Clinch  on  its  northern  shore. 

You  will  begin  somewhere  on  Range  “D”.  Remember,  you  will  be  able  to  call  up  the  chart  and  a 
course  card  like  this  from  the  bridge. 

•  When  we  ask  you  to  report  a  contact,  please  give  a  detailed  report  including:  range,  bearing, 
AOB,  and  your  concern  about  the  contact. 

Any  Questions?  Let’s  go  to  the  bridge  mock-up. 
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APPENDIX  I:  VESUB  TEE  SCENARIO  SCRIPT  (INBOUND  FIRST) 

(Continued) 

VESUB  TEE  Scenario  Events 
Scenario  3  -  Testing  (Kings  Bay  Outboiind)(page  1  of  4) 


Event 

Time 

Component 

Instructor  Script 

1.  Sub  is  placed  on 
Range  “E”.  SOOvds 
from  the  intersection  of 
Range  ‘‘D”.  50vds 
right  of  track.  Heading: 
152 

System  in  Frozen 
Mode 

0:00 

IP.  Locating  &  ID 
Navigation  aids;  2P. 
Judging  Distance; 

2C.  Understanding 
relationship  of  visual 
cues  to  chart(s) 

•  As  quickly  and  accurately  as  possible 
tell  us  where  you  are  now  located? 

•  What  did  you  use  to  determine  your 
location? 

•  What  is  your  distance  left  or  right  of 
track?  (if  not  included  above) 

•  How  did  you  determine  your  distance 

L/R  of  track?  (if  not  included  above) 

2.  Trainee  is  asked  to 
report  all  visible 
contacts. 

7C.  Prioritize  contacts; 

2P.  Judging  distance 

Please  report  all  visible  contacts.  For 

NOW  if  vou  want  binocular  view,  ask  the 
instructor.  After  we  start  the  scenario, 
you  will  be  able  to  obtain  binoc  view  by 
saying  “binoculars”  into  the  microphone. 

3.  Go  to  Run 

4,  Trainee  receives 

FULL  report  from  the 
Navigator 

0:10 

8P.  Detecting  comms. 

•  Instructor  initiates  reports  from  the 

Nav:  USE  SYSTEM 

•  distance  from  track 

•  recommended  course 

•  recommended  speed 

•  sounding 

•  set  &  drift 

•  distance  to  turn 

•  next  course 

5,  Trainee  orders  ship 
to  get  underway  and 
adjusts  course 

*  IOC.  Correctly  using 
ship’s  systems; 

12C.  Effective 
communication 

Instmctor: 

“Bridge,  Navigator.  Recommend  you 
make  turns  for  12  knots” 

6.  Trainee  commands 
turn  to  Range  D  after 

Nav  marks  turn 

2:00 

12C.  Effective 
communication 

Go  to  the  Next  Page 
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APPENDIX  I:  VESUB  TEE  SCENARIO  SCRIPT  (INBOUND  FIRST) 

(Continued) 

VESUB  TEE  Scenario  Events  (Continued) 

Scenario  3  -  Testing  (Kings  Bay  Outbound)(Continued  -  page  2  of  4) 


Event 

Time 

Component 

Instructor  Script 

7.  Trainee  orders  steady 
on  recommended 
course  (171) 

2:30 

IOC.  Correctly  using 
ship’s  systems; 

12C.  Effective 
communication 

8.  FULL  Navigator 
Report  AFTER 

STEADY  ON  171 

3:00 

8P.  Detecting 
communications 

System  generated  Nav  report: 

•  distance  from  track 

•  recommended  course 

•  recommended  speed 

•  sounding 

•  set  &  drift 

•  distance  to  turn 

•  next  course 

9,  Man  Overboard 
(trainee  hears  report 
from  lookout  played  by 
instructor) 

5:00 

8P.  Detecting  comms. 
lie.  Taking  corrective 
actions 

12C.  Effective  comms. 

“OOD,  Lookout.  Man  overboard,  port 
side.” 

10.  Report  from 

Control  (instructor) 

6:00 

8P.  Detecting  comms. 

•  Instructor  says  “Bridge,  Navigator. 

Have  contacted  C-Tractor.  He  has  agreed 
to  recover  man  overboard.  Recommend 
returning  to  base  course  171  at  ahead  2/3” 

11.  Trainee  gets  ship 
xmderway  again 

7:00 

IOC.  Correct  operation 
of  ship’s  systems 

12C.  Effective  comms. 

12.  Nav  reports 
incorrect  position  and 

8:00 

lie.  When  &  how  to 
take  corrective  actions 
12C.  Effective  comms. 

Instructor  says: 

•  If  center  -  “Hold  you  left  of  track. 
Recommend  coming  right  to  180  to 
regain  track. 

•  If  left  -  hold  you  right  of  track. 
Recommend  coming  R  192 

•  If  right  -  hold  you  left  of  track. 
Recommend  192 

incorrect  course  to 
regain  track.  (Left  when 
Right  or  Right  when 

Left) 

8:15 

Go  to  the  Next  Page 

1-16 


Technical  Rqjort  98-003 


APPENDIX  I:  VESUB  TEE  SCENARIO  SCRIPT  (INBOUND  FIRST) 

(Continued) 

VESUB  TEE  Scenario  Events  (Continued) 

Scenario  3  -  Testing  (Kings  Bay  OutboundV Continued  -  page  3  of  4) 


Event 

Time 

Component 

Instructor  Script 

14.  Trainee  commands 
turn  to  Ran^e  C 

Course:  184 

12:00 

12C.  Effective  comms. 

15.  FULL  Nav  report 
(after  steady  on  184) 

8P.  Detecting  comms. 

System  generated  Nav  report: 

•  distance  from  track 

•  recommended  course 

•  recommended  speed 

•  sounding 

•  set  &  drift 

•  distance  to  turn 

•  next  course 

16.  Trainee  sees  Ferry 
ahead 

13:00 

5C.  Rules  of  Road 

12C.  Effective  comms. 

17.  Instructor  controls 

Ferry  for  Port-to-port 
passage 

15:00 

18.  Trainee  gives 
confimands  to  make 
port-to-port  passage 

5C.  Rules  of  Road 

12C.  Effective  comms. 

19.  Trainee  increases 
speed 

16:30 

20.  Trainee  commands 
turn  to  Range  B 

Course:  170 

19:00 

12C.  Effective  comms. 

21.  SHORT  Nav 

Report  (after  steady  on 
170) 

8P.  Detecting  comms. 

System  Generates: 

•  distance  to  turn 

•  next  course 

•  distance  from  track 

22.  Trainee  commands 
turn  to  Range  A-2 
Course:  151.5 

12C.  Effective  comms. 

23.  SHORT  Navigator 
Report  (after  steady  on 
151) 

i  8P.  Detecting  comms. 

System  Generates: 

•  distance  to  turn 

•  next  course 

•  distance  from  track 

Go  to  the  Next  Page 

1-17 


Technical  Report  98-003 


APPENDIX  I:  VESUB  TEE  SCENARIO  SCRIPT  (INBOUND  FIRST) 

(Continued) 

VESUB  TEE  Scenario  Events  (Continued) 

Scenario  3  -  Testing  (Kings  Bay  Outbound!  (Continued  -  page  4  of  4) 


Event 

Time 

Component 

Instructor  Script 

24.  Trainee  commands 
turn  to  Ran^e  A-1  after 
Nav  Marks  Turn 

Course:  122.5 

22:00 

12C.  Effective  comms. 

25.  SHORT  Navigator 
Report  (after  steady  on 
122) 

8P.  Detecting  comms. 

_  ...  -  . 

System  Generates: 

•  distance  to  turn 

•  next  course 

•  distance  from  track 

26.  Trainee  commands 
turn  to  Range  A  after 
Nav  Marks  Turn 

Course:  114 

24:00 

12C.  Effective  comms. 

27.  FULL  Navigator 
report  (after  steady  on 
114) 

8P.  Detecting  comms. 

System  generated  Nav  report: 

•  distance  from  track 

•  recommended  course 

•  recommended  speed 

•  sounding 

•  set  &  drift 

•  distance  to  turn 

•  next  course 

28.  Yellow  Sounding  - 
trainee  must  take 
correct  actions 

25:30 

8P.  Detecting  comms. 
lie.  Taking  corrective 
actions 

12C.  Effective  comms. 

29.  Trainee  increases 
speed 

26:30 

12C.  Effective  comms. 

Scenario  Ends 
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APPENDIX  J 

VESUB  TEE  PERFORMANCE  DATA  COLLECTION  FORMS 

Scenario  2  -  Training  (Kings  Bay  Inbound) 

(page  1  of  11) 


Participant  #;  6- 
Sroton,  CT 


H 


Xvent 


Tisia 


Performance  Measure 


1.  Own  ship  is  placed  on  Entrance 
Range,  2000yds  from  the  1st  turn, 

25yds  right  of  track. 

While  in  Frozen  Mode 

•  Trainee  is  asked  to  give  location, - 
record  reported  position. 

•  Trainee  is  asked  how  he  determined 
location.-  record  all  info  used. 


00:00 


Left/Right  of  Track i  (POSACRSl) 

•  Reported  Position  (Right  or  Left  of  Track) : 

•  Actual  Position  (Right  or  left  of  Track) : 

•  Difference;  Reported  -  Actual  (POSACRSl) * 


Along  Track:  (POSALMGli  SAT  B  within  200  yds  of  actual  position) 
•  Reported  Position  Along  Track:  (SAT» (1) /UNSAT« (0)  _ 


•  Navigation  aids  used  (check  all  that  apply)  : 


•  Beacon  *E" 


•  Entrance  Range  Front. 


•  Beacon  "F"  _  •  Entrance  Range  Rear 

•  Beacon  "N"  _  •  Ft  Clinch  Front 


2.  Instructor  Intervention 

Go  to  Next  Page 


•  Beacon  "S" 

•  Buoy  "D" 

•  4  Yellow  Buoys 

•  Buoy  *24" (red) 

•  Buoy*25" (green) 


OOiOO 


•  Ft  Clinch  Rear 

•  Ft  Clinch  Flagpole 

•  Amelia  Island 

•  Breakwaters 


Notes; 


POSALNGl:  *  SAT  =  within  200  yds  of  actual  position 


Scenario  2  -  Training  (Kings  Bay  Inbound)  (page  2  of  11) 


Event 

Time 

3.  Trainee  is  asked  to 

00x00 

Actual  Positions 

Reported  Positions 

report  all  visible  contacts. 

Cnt  TBrq  RBrq  Rg  PiDB  Y/N 

Cnt  Response  TBro  RBrq  Rq  AOB  Y/N 

ITug  073  164  50  P20  Y 

ITuq 

7Cbn  117  209  5250  P35  N 

7Cbn 

8Cbn  088  180  6650  0  N 

8Cbn 

llCtl  007  099  2050  S150  N 

llCtl 

3Net  278  010  2420  0  Y 

3Net 

6Slbt  276  008  5450  P15  Y 

6Slbt 

4 .  Instructor  Intervention 

Contacts  ; Found:  Actual  ; 

(ratio  found/actual  =  CNTFNDl) 

Contact  Concern:  #IDed  as  concern/#  possible  of  concern  (CNTCOKl) : 

(CNTCONl  =  ratio  of  items  reported  as  a  concern  /  3) 
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APPENDIX  J:  VESUB  TEE  PERFORMANCE  DATA  COLLECTION  FORMS 

(Continued) 

Scenario  2  -  Training  (Kings  Bay  Inbound) (page  3  o£  11) 


6.  Report  from  Navigator: 


7.  Trainee  orders  ship  to 
get  underway  and  adjusts 
course 

•  "Helm,  Bridge.  All  ahead 
standard" 

•  "Helm,  Bridge.  Come  left 
to  266" 


•  Correct  commands  issued  to  get  underway? (Y/N) 

•  "Helm,  Bridge.  All  ahead  2/3"  _ 

•  "Helm,  Bridge.  Steady  course  266"  _ 

•  Visually  check  rudder?  (Y/N)  _ 

•  Visually  check  ranges?  (Y/N)  _ 

•  Visually  check  bridge  suitcase?  (Y/N)  _ 


•  Number  of  "say  agains" 

•  No  Station  ID 

•  Itiproper  Phraseology 

•  Hesitation 

•  Garbled  Speech 

•  System  Fault 


Total  "Say  Agains"_ 


•  Number  repeated  reports  (failures  to  acknowledge) 


Notes : 

CMDUNWl  =  ratio  of  #  correct  commands  given  to  get  ship  underway  / 
total  #  possible  correct  commands  to  get  ship  underway  (  =  ratio  of  #  correct 
commands  given  to  get  ship  underway  /  6) 


Scenario  2  -  Training  (Kings  Bay  Inbound) (page  4  of  11) 


8.  Man  Overboard  (trainee 
hears  report  from  lookout 
played  by  Instructor) 

•  "Helm,  Bridge.  All 
stop . " 

•  "Maneuvering,  Bridge. 
Stop  and  lock  the  shaft" 

•  "Helm,  Bridge.  "Right 
(left)  full  rudder. 


02i30 I  Start  stop  watch  after  "on  port  tidal” 


•  Time  to  give  command;  30  sec  max  (MOBTimi)  _ 

•  Number  of  correct  actions;  (Time  to  give  command:  30  sec  max); 

•All  Stop  _ 

•Stop  and  lock  the  shaft  _ 

•Left  Full  Rudder  _ 

•Communicated  Casualty  to  Ship  (Y/N) :  _  _ 

•  Visually  check  rudder?  (Y/N)  _ 

•  Visually  check  ranges?  (Y/N)  _ 

•  Visually  check  bridge  suitcase?  (Y/N)  _ 

•  Contact  Tug  (Y/N)  _ 

•  Number  of  "say  agains" 

•  No  Station  ID  _ 

•  Iti^roper  Phrase  _ 

•  Hesitation  _  Total  "Say  Agains* _ 

•  Garbled  Speech  _ 

•  System  Fault  _ 

•  Number  of  repeated  reports  (failures  to  acknowledge)  _ 


Notes : 


MOBTIMEl  =  time  to  begin  issuing  commands  during  man  overboard  event 
MOBCMDSl  =  #  correct  commands  given  during  man  overboard  event 
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APPENDIX  J:  VESUB  TEE  PERFORMANCE  DATA  COLLECTION  FORMS 

(Continued) 

Scenario  2  -  Training  (Kings  Bay  Inbound)  (Continued  -  page  5  of  11) 


Event 

Tima 

9.  Maneuvering  (Instructor) 
reports : 

02:40 

•  Acknowledqe  report?  (Y/N) 

10.  Report  from  control 
(instructor)  -  Tug  is  picking  up 
man  overboard 

•  Aclcnowledqe  report?  (Y/N) 

11.  Trainee  gets  ship  underway 
again 

(Trainee  may  try  to  correct  ship's 
position.  Should  as  a  minimum, 
evaluate  position  and  ask  Kav  for 
backup) 

02:50 

•  Correct  command  (Y/N) 

•  "Helm  Bridqe.  All  ahead  2/3*  (Y/N) 

•  "Helm  Bridqe.  Steady  Course  266"  (Y/N) 

•  Visually  check  rudder?  (Y/N) 

•  Visually  check  ranqes?  (Y/N) 

•  Visually  check  bridqe  suitcase?  (Y/N) 

•  Correct  commands  issued  to  qet  underway? (Y/N)  (CMDUNWl; 

) 

•  "Helm.  Bridqe.  All  ahead  2/3" 

•  "Helm.  Bridqe.  Steady  course  266" 

•  Number  of  repeated  reports  (failures  to  ac)cnowledqe) 

12.  Instructor  Intervention 

Go  to  FREEZE 

13.  Go  to  tiMFREEZE 

Trainee  gets  ship  underway  again 

04il5 

Scenario  2  -  Training  (Kings  Bay  Inbound)  (Continued  -  page  6  of  11) 


Event 

Time 

14.  Trainee  commands  turn  to  Range 

A  after  Nav  marks  turn 

Course  294 

07«30 

•  Marked  turn:  Early  On  time  Late 

•  Correct  command  (Y/N) 

•  "Helm,  Bridqe.  Right  15  degrees  rudder" 

•  "Helm.  Bridge.  Steady  course  294" 

•  Visually  check  rudder?  (Y/N) 

•  Visually  check  ranges?  (Y/N) 

•  Visually  check  bridqe  suitcase?  (Y/N) 

•  Number  of  "say  agains* 

•  No  Station  ID 

•  Improper  Phrase 

•  Hesitation  Total  "Say  Agains" 

•  Garbled  Speech 

•  System  Fault 

•  Number  of  repeated  reports  (failures  to  acknowledge) 

IS.  Nav  reports  incorrect  position 
cuid  incorrect  course  to  regain 
track.  (Left  when  Right  or  Right 
when  Left) 

OStlS 

•  Recognize  Error  (e.q.  Asks  for  correct ion/ "sav  again") (Y/N) 

•  Visually  check  ranges?  (Y/N) 

•  Was  error  self  corrected?  (Y/N) 
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APPENDIX  J:  VESUB  TEE  PERFORMANCE  DATA  COLLECTION  FORMS 

(Continued) 

Scenario  2  -  Training  (Kings  Bay  Inbound)  (Continued  -  page  7  of  11) 


_ Kvent _ Time 

16.  Instructor  Intervention 

17.  Trainee  commands  turn  to  Range  A>1  12i40 

after  Nav  Marks  Turn 

Course I  302.5 


Perfonaance  Measure 


•  Marked  turn:  Early _  On  time _ 

•  Correct  command  (Y/N) 

•  •Helm,  Bridge,  Right  15  degrees  rudder" 

•  "Helm,  Bridge.  Steady  course  302" 


Late 


•  Visually  check  rudder?  (Y/N) 

•  Visually  check  ranges?  (Y/N) 

•  Visually  check  bridge  suitcase?  (Y/N) 


•  Number  of  "say  agains" 

•  No  Station  ID  _ 

•  Improper  Phrase  _ 

•  Hesitation  _  Total  *Say  Agains"_ 

•  Garbled  Speech  _ 

•  System  Fault  _ 

•  Number  of  repeated  reports  (failures  to  acknowledge) 


18.  Nav  reports  correct  information 
(short  report)  after  steady  on  302 

19.  Trainee  commands  turn  to  Range  A- 2 
after  Nav  Marks  Turn 

Course I  331.5 


»  Acknowledge  report?  (Y/N) 

•  Marked  turn:  Early _  On  time _ 

•  Correct  command  (Y/N) 

•  "Helm,  Bridge .  Right  15  degrees  rudder’ 

•  'Helm,  Bridge.  Steady  course  331" 


Late 


•  visually  check  rudder?  (Y/N) 

•  Visually  check  ranges?  (Y/N) 

•  Visually  check  bridge  suitcase?  (Y/N) 


•  Number  of  "say  agains" 

•  No  Station  ID  _ 

•  Improper  Phrase  _ 

•  Hesitation  _  Total  *Say  Agains", 

•  Garbled  Speech  _ 

•  System  Fault  _ 


Go  to  the  Next  Page 


•  Number  of  repeated  reports 


(failures  to  acknowledge) 


Scenario  2  -  Training  (Kings  Bay  Inbound) (Continued  -  page  8  of  11) 


Xvent 

Time 

Performance  Measure 

20.  Navigator  Short  Report  (after 
steady  on  331) 

•  Acknowledge  report?  (Y/N) 

21.  Trainee  coinnands  turn  to  Range  B 
after  Nav  Marks  Turn 

Course!  350 

•  Marked  turn:  Early  On  time  I.atc 

•  Correct  command  (Y/N) 

•  'Helm,  Bridge,  Right  15  degrees  rudder" 

•  •Helm,  Bridge .  Steady  course  350" 

•  Visually  check  rudder?  (Y/N) 

•  Visually  check  ranges?  (Y/N) 

•  Visually  check  bridge  suitcase?  (Y/N) 

•  Number  of  “say  agains" 

•  No  Station  ID 

•  Improper  Phrase 

•  Hesitation  Total  "Sav  Agains" 

•  Garbled  Speech 

•  System  Fault 

•  Number  of  repeated  reports  (failures  to  acknowledge) 

22.  Navigator  short  report  (after 
steady  on  350) 

•  Acknowledge  report?  (Y/N) 

23.  Ferry  Becomes  visible  (possible) 

15;30 

Reouest  report  from  Contact  Coordinator  (Y/N) ; 

Reported  Ferry  (Y/N) : 

Asks  for  Bridge  to  Bridge  communications  (Y/N) : 

Go  to  the  Next  Page 
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APPENDIX  J:  VESUB  TEE  PERFORMANCE  DATA  COLLECTION  FORMS 

(Continued) 

Scenario  2  Training  (Kings  Bay  Inbound)  (Continued  -  page  9  of  11} 


Zvent 

Time 

24.  Trainee  coiumands  turn  to  Ranoe  C 
after  Nav  Marks  Turn 

Course:  004 

17*30 

•  Marked  turn:  Early  On  time  Late 

•  Correct  command  (Y/N) 

•  •Helm.  Bridae.  Riqht  IS  decrees  rudder" 

•  "Helm,  Bridcre.  Steady  course  004" 

•  Visually  check  rudder?  (Y/N) 

•  Visually  check  ranqes?  (Y/N) 

•  Visually  check  bridqe  suitcase?  (Y/N) 

•  Number  of  "say  agains" 

•  No  Station  ID 

•  Iraorooer  Phrase 

•  Hesitation  Total  "Say  Aqains" 

•  Garbled  Speech 

•  System  Fault 

•  Number  of  reoeated  reports  (failures  to  acknowledqe) 

25.  Navigator  Full  report  (after 
steady  on  004 

•  Acknowledqe  report?  (Y/K) 

26.  Contact  Coordinator  Report 

Nav  Report 

•  Acknowledqe  report?  (Y/N) 

§■■■■ 

Scenario  2  -  Training  (Kings  Bay  Inbound)  (Continued  -  page  10  of  11) 


Event 

Time 

27.  OOD  maneuvers  own  ship  to  right 
for  P  to  P  passage.  Instructor 
controls  Perry  for  Port-to-port 
passage 

20*00 

•  Moves  own  ship  to  riqht  (Y/N) 

•  Slows  own  ship  when  passinq?  (Y/N) 

28.  Trainee  increases  speed 

21:00 

•  Correct  commands  issued  to  get 

underway? (Y/N)  (CKDUNWl:  ) 

•  "Helm,  Bridqe.  All  ahead  2/3" 

•  "Helm.  Bridqe.  Steady  course  004" 

•  Number  of  "say  agains" 

•  No  Station  ID 

•  Improper  Phrase 

•  Hesitation  Total  "Say  Aqains" 

•  Garbled  Speech 

•  System  Fault 

29.  Go  to  FREEZE 

Instructor  Intervention 

30.  Yellow  sounding 

23:30 

■■■■■ 
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APPENDIX  J:  VESUB  TEE  PERFORMANCE  DATA  COLLECTION  FORMS 

(Continued) 

Scenario  2  -  Training  (Kings  Bay  Inbound) (Continued  -  page  11  of  11) 


32.  Instructor  (Nav)  gives  info, 
if  trainee  does  not  request 
within  30  secs. 


34.  Go  to  UN7RSKZS. 

Trainee  increases  speed  &  sets 


•  Time  to  Give  Command 


•  Slows  own  ship?  (Y/N)  _ 

ExaiT^le:  *Helm,  Bridge.  All  stop*  YELWSNDls  _ 

•  Checked  ranges?  (Y/N)  _  tmmmmmmmmmmmmmmmmmmmmm 

•  Fixes  ship's  position  _ 

Example:  'Navigator,  Bridge.  What  is  current  position? 

•  Requests  backup  sounding?  (Y/N)  _ 

Example:  "Navigator,  Bridge.  Report  (request)  backup  sounding* 

•  Requests  bearing  to  best  water  or  center  of  channel?  (Y/N)  _ 

Example:  "Navigator,  Bridge.  What  is  the  direction  to  good  water?* 
Example:  "Navigator,  Bridge.  What  is  the  nearest  shoal  water? 

•  Reports  to  CAPT?  (Y/N)  _ 


•  Does  Instructor  have  to  provide  info?  (Y/N) 


24:30  •  Order  bell 


•  Number  of  "say  agains* 

•  No  Station  ID  _ 

•  Improper  Phrase  _ 

•  Hesitation  _ 

•  Garbled  Speech  _ 

•  System  Fault  _ 


to  regain  control  of  vessel 


Total  "Say  Agains*__ 


35.  Scenario  Ends 


Notes:  YELWSNDl  =  #  correct  commands  given  during  yellow  sounding  event 


Scenario  3  -  Testing  (Kings  Bay  Outbound) (page  1  of  10) 


1.  Sub  is  placed  on  Range  "K* , 
800yds  from  the  intersection  of 
Range  *D*.  50yds  right  of  track. 
Heading:  152 

While  in  Frozen  Mode 

•  Trainee  is  asked,  "Where  are 
you?"  He  answers  -  record 
reported  position. 

•  Trainee  is  asked,  "What 
information  did  you  use  to 
determine  your  position?*  He 
answers  -  record  all  info  used. 


Left/Right  of  Tracks  (POSACR52) 

•  Reported  Position (  Right  or  Left  of  Track  ) :  _ 

•  Actual  Position  (Right  or  Left  of  Track  ) :  _ 

•  Difference:  Reported  -  Actual  (P0SXC31S2)  :  _ 

Along  Tracks  (P0SALNG2  t  SAT  ■  within  200  yds  of  actual  position) 

•  Reported  Position  Along  Track:  (SAT  (1) /UNSAT  (0)) _ 

•  Navigation  aids  used  (check  all  that  apply) : 


•  Buoy  49 


•  Upper  "E*  Fronts 


•  Upper  "E*  Rear 


•  *D*  Front 


•  Lower  "E*  Front 


•  Lower  *E*  Rear 


Go  to  the  Next  Page 


Notes:  POSALNG2:  *  SAT  =  within  200  yds  of  actual  position 
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APPENDIX  J:  VESUB  TEE  PERFORMANCE  DATA  COLLECTION  FORMS 

(Continued) 


Scenario  3  -  Testing  (Kings  Bay  Outbound) (Continued  -  page  2  of  10) 


Event 

Time 

2.  Trainee  is  asked 

0(00 

Reported  Positions 

to  report  all  visible 

contacts . 

Cnt  Tbro  Rbrq  ^  Y/N 

Cnt  Response  TrBro  RlBra  Ra  AOB 

Y/N 

1  Tug  187  034  180  P145  Y 

1  Tuq 

__  _ 

2  Tug  344  192  180  SIS  Y 

2  Tuq 

_  _ 

7  CbnCru  166  014  3200  S168  Y 

7  CbnCr 

__  _ 

11  YP689  320  167  2200  S20  N 

11  YP689 

_  _ 

10  sail  065  272  900  P45  N 

10  sail 



12*SSBN*  181  029  300  P175  N 

12*SSBN* 

-  - 

#  Contacts:  Found  Actual 

(ratio  found/ actual  >  CNTFND2) 

Contact  Concern:  #  IDed  as  concern/#  possible  of  concern 

(CNTCON2) ( 

(  CNTCON2  m  ratio  Of  items  reported  as  a  concern  /  3) 

1 

0:00 

HHHIi 

1 

1 

■■HU 

—Mi 

Scenario  3  -  Testing  (Kings  Bay  Outbound) (Continued  -  page  3  of  10) 


♦ 


_ Event _ 

4.  Trainee  receives 

Full  reports  from  the 
Navigator _ 

5.  Trainee  orders  ship 
to  get  underway  and 
adjusts  course 


0:10 


•  Acknowledge  report? 


(V/N) 


Performance  Measure 


0t30 


•  Correct  commands  issued  to  get  underway? 

•  “Helm,  Bridge.  All  ahead  2/3"  _ 

•  "Helm,  Bridge.  Come  Left  to  -  _ 


(Y/N) 


(CMDUNW2 


•  Visually  check  rudder?  (Y/N) 

•  Visually  check  ranges?  (Y/N) 

•  Visually  check  bridge  suitcase?  (Y/N) 


•  Number  of  *say  agains" 

•  No  Station  ID  _ 

•  Ittqproper  Phrase  _ 

•  Hesitation  Total  *Say  Agains", 

•  Garbled  Speech  _ 

•  System  Fault 


6 .  Trainee  commands 
turn  to  Range  D  after 
Nav  marks  turn,  steady 

on  recommended  course 

(171) 


•  Number  of  repeated  reports  (failures  to  acknowledge) 


•  Marked  Turn:  Early _  On  time  _  Late_ 

•  '‘Helm,  Bridge.  RT  IS  degrees  rudder" 

•  "Helm,  Bridge.  Steady  course  171"  _ 

•  Visually  check  rudder?  (Y/N)  _ 

•  Visually  check  rcuiges?  (Y/N)  _ 

•  Visually  check  bridge  suitcase?  (Y/N)  _ 

•  Number  of  "say  agains" 

•  No  Station  ID  _ 


•  In^roper  Phrase  _ 

•  Hesitation  _  Total  *Say  Agains", 

•  (Sarbled  Speech  _ 


7.  Trainee  orders 
steady  on  recommended 
course  (171) 


2(00 


•  System  Fault  _ 

•  Number  of  repeated  reports  (failures  to  acknowledge) 


) 


Notes :  CMDUND2  =  ratio  of  #  correct  commands  given  to  get  ship  underway  / 
total  #  possible  correct  commands  to  get  ship  underway  (=  ratio  of 
#  correct  commands  given  to  get  ship  underway  /  6) 
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APPENDIX  J:  VESUB  TEE  PERFORMANCE  DATA  COLLECTION  FORMS 

(Continued) 

Scenario  3  -  Testing  (Kings  Bay  Outbound)  (Continued  -  page  4  of  10) 


_ »vent _ 

8.  Full  Navigator  Report 
after  steady  on  171 


Tine 

3t00 


•  Acknowledge  report? 


(Y/N) 


Performance  Measure 


9.  Man  Overboard 
(trainee  hears  report 
from  lookout  played  by 
instructor) 


SfOO 


start  stop  watch  after  ''on  port  side  I* 

•  Time  to  give  command;  30  sec  max  (M0BTI1OC2) » 


•  Number  of  correct  actions: 


•  "Helm,  Bridge.  All 
Stop . * 

•  "Maneuvering, 
Bridge.  Stop  and 
lock  the  shaft." 

"Helm,  Bridge.  Left  full 
rudder . " 


•  All  Stop  (Y/N) : 

•  Stop  and  lock  the  shaft  (Y/N) : 

•  Left  Pull  Rudder  (Y/N) : 

•  Communicated  Casualty  to  Ship (Y/N): 


•  Visually  check  rudder?  (Y/N)  _  lMOBcaiDS2i 

•  Visually  check  ranges?  (Y/N)  _ 

•  Visually  check  bridge  suitcase?  (Y/N)  _ 

•  Contact  Tug  (Y/N)  _ 


•  Number  of  "say  agains" 

•  No  Station  ID  _ 

•  In^roper  Phrase  _ 

•  Hesitation  _  Total  "Say  Agains" 

•  Garbled  Speech  _ 

•  System  Fault  _ 


•  Number  of  repeated  reports  (failures  to  acknowledge) 


Notes : 


MOBCMDS2  =  #  correct  commands  given  during  man  overboard  event 
MOBTIME2  =  time  to  begin  issuing  commands  during  man  overboard  event 


Scenario  3  -  Testing  (Kings  Bay  Outbound)  (Continued  -  page  5  of  10) 


Bvant 

1  Tiaie 

10.  Report  from  Control 
(instructor)  -  Tug  is  picking 
up  man  overboard 

m 

•  Acknowledge  report?  (Y/N) 

11.  Trainee  gets  ship 
underway  again 

•  Correct  command  (Y/N) 

•  Correct  commands  issued  to  get  underway?  (Y/N)  (CMDXTNnflr  ) 

•  "Helm,  Bridge.  All  ahead  2/3" 

•  "Helm,  Bridge.  Steady  course  171" 

•  Visually  check  rudder?  (Y/N) 

•  Visually  check  ranges?  (Y/N) 

•  Visually  check  bridge  suitcase?  (Y/N) 

•  Number  of  "say  agains" 

•  No  Station  ID 

•  Improper  Phrase 

•  Hesitation  Total  "Say  Agains" 

•  Garbled  Speech 

•  System  Fault 

•  Number  of  repeated  reports  (failures  to  acknowledge) 
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APPENDIX  J:  VESUB  TEE  PERFORMANCE  DATA  COLLECTION  FORMS 

(Continued) 


Scenario  3  -  Testing  (Kings  Bay  Outbound)  (Continued  -  page  6  of  10) 


12.  Nav  reports  incorrect  8:00 

position  and  incorrect 
course  to  regain  track. 

(Left  when  Right  or  Right 
when  Left) 


•  Acknowledge  report?  (Y/N)  _ 

•  Marked  turn:  Early _  On  time _  Late _ 

•  Correct  command  (Y/N) 

•  *Helm,  Bridge.  Right  15  degrees  rudder"  _ 

•  '‘Helm,  Bridge.  Steady  course  184"  _ 

•  Visually  check  rudder?  (Y/N)  _ 

•  Visually  check  ranges?  (Y/N)  _ 

•  Visually  check  bridge  suitcase?  (Y/N)  _ 

•  Number  of  “say  agains" 

•  No  Station  ID  _ 

•  In^jroper  Phrase  _ 

•  Hesitation  _  Total  '‘Say  Agains* _ 

•  Garbled  Speech  _ 

•  System  Fault  _ 

•  Number  of  repeated  reports  (failures  to  acknowledge)  _ 


•  Recognize  Error  (Asks  for  correction  or  "Say  Again")  (Y/N) 

•  Visually  check  ranges?  (Y/N) 

•  Was  error  self  corrected?  (Y/N)  _ 


Scenario  3  -  Testing  (Kings  Bay  Outbound)  (Page  7  of  10) 


15.  Full  Nav  report  (after 
steady  on  184) 


16.  Trainee  sees  Ferry  ahead 


17 .  Instructor  controls 
Perry  for  Port-to-port 
passage 


18.  Trainee  gives  commands 
to  make  port-to-port  passage 


19.  Trainee  increases  speed 


Go  to  the  Next  Page 


Perfomance  Measure 


Acknowledge  report?  (Y/N) 


Request  report  from  contact  coordinator  (Y/N) 

•  Reports  Ferry  as  a  concern  (Y/N) :  _ 

•  Asks  for  Bridge  to  Bridge  Communications  (Y/N) 


•  Moves  own  ship  to  right  (Y/N)  _ 

•  Slows  own  ship  when  passing?  (Y/N) 


Correct  Command?  (Y/N) :  _ 

"Helm,  Bridge.  All  ahead  standard" 

Returns  to  base  course  (Y/N)  ; 

•  Number  of  "say  agains" 

•  No  Station  ID  _ 

•  Improper  Phrase  _ 

•  Hesitation  _ 

•  Garbled  Speech  _ 

•  System  Fault  _ 


Total  "Say  Agains" 


•  Number  of  repeated  reports  (failures  to  acknowledge) 


•  Visually  check  rudder?  (Y/N) 

•  Visually  check  reuiges?  (Y/N) 

•  Visually  check  bridge  suitcase?  (Y/N) 
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APPENDIX  J:  VESUB  TEE  PERFORMANCE  DATA  COLLECTION  FORMS 

(Continued) 

Scenario  3  -  Testing  (Kings  Bay  Outbound)  (Continued  -  page  8  of  10) 


»vent _ 

20.  Go  to  UM7RXSZS 
Trainee  commands  turn  to 
Range  B  after  Nav  Marks  Turn 

Course;  170 


21.  Short  Navigator  Report 

(after  steady  on  170) _ 

22.  Trainee  commands  turn  to 
Range  A- 2  after  Nav  Marks 
Turn 

Course:  ISl.S 


Time  Performance  Meaaure 

19*00  •  Marked  turn;  Early _  On  time _  Late _ 

•  Correct  command  (Y/N) 

•  •Helm,  Bridge.  Left  15  degrees  rudder"  _ 

•  •Helm,  Bridge.  Steady  course  170"  _ 

•  Visually  check  rudder?  (Y/N)  _ 

•  Visually  check  ranges?  (Y/N)  _ 

•  Visually  check  bridge  suitcase?  (Y/N)  _ 

•  Number  of  •say  agains" 

•  No  Station  ID  _ 

•  Improper  Phrase  _ 

•  Hesitation  _  Total  "Say  Agains" 

•  Garbled  Speech  _ 

•  System  Fault  _ 

_ •  Number  of  repeated  reports  (failures  to  acknowledge) 

•  Acknowledge  report?  (Y/N)  _ 


•  Marked  turn;  Early _  On  time _  Late _ 

•  Correct  command  (Y/N) 

•  •Helm,  Bridge.  Left  15  degrees  rudder"  _ 

•  •Helm,  Bridge.  Steady  course  151"  _ 

•  Visually  check  rudder?  (Y/N)  _ 

•  Visually  check  ranges?  (Y/N)  _ 

•  Visually  check  bridge  suitcase?  (Y/N)  _ 

•  Number  of  •say  agains' 

•  No  Station  ID  _ 

•  iTt^roper  Phrase  _ 

•  Hesitation  _  Total  "Say  Agains"^ 

•  Garbled  Speech  _ 

•  System  Fault  _ 

•  Number  of  repeated  reports  (failures  to  acknowledge) 


Go  to  the  Next  Page 


Scenario  3  -  Testing  (Kings  Bay  Outbound) (Continued  -  page  9  of  10) 


Performance  Measure 


23.  Short  Navigator  Report 
(after  steady  on  151) _ 


•  Acknowledge  report?  (Y/N) 


24.  Trainee  commands  turn  to 
Range  A«1  after  Nav  Marks 
Turn  Course;  122.5 


On time 


•  Marked  turn:  Early _  _ 

•  Correct  command  (Y/N) 

•  •Helm,  Bridge.  Left  15  degrees  rudder" 

•  •Helm,  Bridge.  Steady  course  122" 

•  Visually  check  rudder?  (Y/N) 

•  Visually  check  ranges?  (Y/N) 

•  Visually  check  bridge  suitcase?  (Y/N) 

•  Number  of  •say  agains" 

•  No  Station  ID  _ 

•  Improper  Phrase  _ 

•  Hesitation  _ 

•  Garbled  Speech  _ 

•  System  Fault  _ 

•  Number  of  repeated  reports  (failures  to  acknowledge) 


Total  •Say  Agains" 


25.  Short  Navigator  Report 
(after  steady  on  122 _ 


•  Acknowledge  report?  (Y/N) 


26.  Trainee  commands  turn  to 
Range  A  after  Nav  Marks  Turn 

Course;  114 


•  Marked  turn:  Early _  On  time _  Late _ 

•  Correct  command  (Y/N) 

•  •Helm,  Bridge.  Left  15  degrees  rudder"  _ 

•  "Helm,  Bridge.  Steady  course  114"  _ 

•  Visually  check  rudder?  (Y/N)  _ 

•  Visually  check  ranges?  (Y/N)  . 

•  Visually  check  bridge  suitcase?  (Y/N)  _ 

•  Number  of  •say  agains' 

•  No  Station  ID  _ 

•  Improper  Phrase  _ 

•  Hesitation  _  Total  'Say  Agains"_ 

•  Garbled  Speech  _ 

•  System  Fault 

•  Number  of  repeated  reports  (failures  to  acknowledge) 
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APPENDIX  J:  VESUB  TEE  PERFORMANCE  DATA  COLLECTION  FORMS 

(Continued) 

Scenario  3  -  Testing  (Kings  Bay  Outbound) (Continued  -  page  10  of  10) 


Event 

Time 

28.  Pull  Navigator  Report 
(after  steady  on  114 

•  Acknowledge  report?  (Y/N) 

29.  Yellow  Soundinq  -  trainee 

25i00 

•  Ti«*  to  give  command 

must  take  correct  actions 

•  Slows  own  shin?  (Y/N) 

Examples  *Helra,  Bridge.  All  stop* 

•  Checked  ranges?  (Y/N) 

•  Fixes  ship's  position 

Example:  "Navigator,  Bridge.  What  is  cur 

•  Reouests  backup  soundinq?  (Y/N) 

Exaiiples  "Navigator,  Bridge.  Report  (re 

•  Requests  bearing  to  best  water  or  center  oJ 

Example:  "Navigator,  Bridge.  What  is  thi 
Example:  "Navigator,  Bridge.  What  is  tb 

•  Reports  to  CAPT?  (Y/N) 

YBLSMD2 : 

rent  position? 

quest)  backup  sounding* 

channel?  (Y/N) 
e  direction  to  good  water?" 
e  nearest  shoal  water? 

33.  Trainee  increases  speed 

26:00 

•  Ordered  bell?  (Y/N) 

•  Ordered  Course?  (Y/N) 

Notes : 

YELWSND2  =  #  correct  commands  given  during  yellow  sounding  event 
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APPENDIX  K 

RESULTS  OF  VESUB  TEE  SIMULATION  SIDE  EFFECTS  ASSESSMENT 


Ss# 

GenDis 

Fatigue 

HdAche 

EyeStrn 

DifFoc 

Saliv 

Sweat 

Nausea 

N-1 

1 

0 

0 

0 

0 

0 

0 

0 

N-2 

b 

0 

0 

0 

I 

0 

0 

0 

N-3 

0 

0 

0 

0 

0 

0 

0 

0 

'N-4 

0 

0 

0 

0 

0 

"  '  b 

0 

N-5 

0 

0 

0 

0 

0 

0 

0 

0 

N-6 

0 

0 

0 

0 

0 

0 

0  ' 

'"'o'  ' 

N-7 

0 

1 

0 

0 

0 

0 

0 

0 

N-8' " 

0 

6 

0 

0 

0 

0 

N-9 

0 

1 

0 

0 

0 

0 

1 

0 

N-lb 

1 

1 

1 

0 

0 

b 

0 

0 

N-11 

1 

1 

0 

0 

0 

0 

1 

0 

N-12 

^  O'  • 

0 

0 

0 

0 

0 

0 

N-1 3 

0 

0 

0 

0 

0 

0 

0 

0 

N-i4 

0 

o' 

iilBIIE' 

0 

0 

0 

0  r 

N-1 5 

0 

o' 

0 

0 

0 

0 

0 

0 

N-1 6 

0 

0 

0 

0  ^ 

0 

0 

N-1 7 

0 

0 

0 

0 

0 

0 

0 

0 

N-1 8 

O' 

0 

0 

0 

0 

N-1 9 

0 

0 

0 

0 

0 

0 

0 

0 

;n-20 

0 

0 

0 

0 

0 

6 

.  Q  . 

ilifllllii 

Note:  N  =  Norfolk;  G  =  Groton 

0  =  None;  1  =  Slight;  2  =  Moderate;  3  =  Severe 
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Appendix  K: 

RESULTS  OF  VESUB  TEE  SIMULATION  SIDE  EFFECTS  ASSESSMENT 

(Continued) 


Ss# 


DifCon 


N-1 

N-2 

N-3 

N-4 

N-5 

N-6 

N-7 

N-8 

N-9 

N-10 

N-11 

N-1 2 

N-1 3 

N-14 

N-1 5 

N-16 

N-17 

N-1 8 


0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 


N-1 9  0 

N-20  0 


Symptom  Scores  Prior  to  Orientation  Scenario _ 

FuIIHd  BlurVis  Diz(0)  Diz(C)  Vertigo  Stomac  Burping 


0  0  0 
0  1  0 
0  0  0 
0  0  0 
0  0  0 
0  0  0 
0  0  0 
0  0  0 
0  0  0 
0  0  0 
0  0  0 
0  0  0 
0  0  0 
0  0  0 
0  0  0 
0  0  0 
0  0  0 
0  0  0 
0  0  0 
0  0  0 


0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 


0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 


0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 


0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 


Note:  N  =  Norfolk;  G  =  Groton 

0  =  None;  1  =  Slight;  2  =  Moderate;  3  =  Severe 


K-2 


Technical  Report  98-003 


APPENDIX  K: 

RESULTS  OF  VESUB  TEE  SIMULATION  SIDE  EFFECTS  ASSESSMENT 

(Continued) 

Symptom  Scores  Prior  to  Orientation  Scenario  _ 


Ss# 

GenDis 

Fatigue 

HdAche 

EyeStrn 

DifFoc 

Saliv  Sweat 

Nausea 

G4 

0 

1 

0 

w:.,  1 

0 

0 

0 

G-2 

1 

0 

0 

0 

0 

0 

0 

0 

G-3 

0 

0 

0 

0 

. io  WW 

0 

0 

G-4 

0 

0 

0 

0 

0 

0 

0 

0 

G-5 

. c:.:: 

2 . 

0 

0 

,  1 

0 

0 

G-6 

0 

0 

0 

0 

0 

0 

0 

b 

G-7 

■  '"1 

,  1 

1 

0 

0 

0 

0 

0 

G-8 

0 

0 

0 

0 

0 

0 

0 

0 

■  G-9 

1 

0 

6 

0 

0 

0 

0 

G-10 

0 

0 

0 

0 

0 

1 

1 

0 

G-11 

0  , 

0 

0 

0 

0 

0 

0 

G-12 

0  * 

0 

0 

0 

0 

0 

0 

0 

G-13 

1 

0 

0 

0 

0 

0 

b  : 

b 

G-14 

0 

0 

0 

0 

0 

0 

0 

0 

G-15 

0 

'  1 

0 

■  .’o  ■ ' ' 

0 

0 

0 

0 

G-16 

0 

0 

0 

0 

0 

0 

0 

0 

"G-17' 

0 

o' 

0 

0 

0  ' 

■  •"  -0' 

0  V 

. o'  ■  ‘ 

G-18 

0 

0 

0 

0 

0 

0 

0 

0 

G-19 

0  '  ■- 

1 

0 

'  0 

0 

0  ■/. 

W[W'' 

0 

G-20 

1 

1 

0 

0 

0 

0 

0 

0 

'  G-21 

0 

l" 

0 

b 

0 

i' 

b 

G-22 

0 

1 

0 

0 

0 

0 

0 

0 

Note:  N  =  Norfolk;  G  =  Groton 

0  =  None;  1  =  Slight;  2  =  Moderate;  3  =  Severe 
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APPENDIX  K: 

RESULTS  OF  VESUB  TEE  SIMULATION  SIDE  EFFECTS  ASSESSMENT 

(Continued) 


Symptom  Scores  Prior  to  Orientation  Scenario 


Ss# 

DifCon 

FullHd 

BlurVis 

Diz(0 

Vertigo 

Stomac 

Burpinj 

G-1 

0 

0 

1 

0 

0 

0 

0 

0 

G-2 

0 

0 

0 

0 

0 

0 

0 

0 

G-3 

0 

0 

0 

0 

0 

0 

0 

0 

G-4 

0 

0 

0 

0 

0 

0 

0 

0 

G-5 

0 

0 

0 

0 

0 

0 

0 

0 

G-6 

0 

0 

0 

0 

0 

0 

0 

0 

G-7 

0 

1 

0 

0 

0 

0 

0 

0 

G-8 

0 

0 

0 

0 

0 

0 

0 

0 

G-9 

0 

1 

0 

0 

0 

0 

0 

0 

G-10 

0 

0 

0 

0 

0 

0 

1 

0 

G-11 

0 

0 

0 

0 

0 

0 

0 

0 

G-12 

0 

0 

1 

0 

0 

0 

0 

0 

G-13 

0 

1 

0 

0 

0 

0 

0 

0 

G-1 4 

0 

0 

0 

0 

0 

0 

1 

0 

G-1 5 

0 

0 

0 

0 

0 

0 

0 

0 

G-16 

0 

0 

0 

0 

0 

0 

0 

0 

G-17 

0 

0 

0 

0 

0 

0 

0 

0 

G-1 8 

0 

0 

0 

0 

0 

0 

0 

0 

G-19 

1 

0 

0 

0 

0 

0 

0 

0 

G-20 

0 

2 

0 

0 

0 

0 

0 

0 

G-21 

0 

0 

0 

0 

0 

0 

0 

0 

G-22 

0 

0 

0 

0 

0 

0 

0 

0 

Note:  N  =  Norfolk;  G  =  Groton 

0  =  None;  1  =  Slight;  2  =  Moderate;  3  =  Severe 
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APPENDIX  K: 

RESULTS  OF  VESUB  TEE  SIMULATION  SIDE  EFFECTS  ASSESSMENT 

(Continued) 


Trans 


formed  Scores  Prior  to  Orientation  Scenario 


Ss# 

Tot“n” 

Tot‘‘o” 

Tof’d” 

«N« 

“O” 

N-1  ' 

0 

1 

0 

0 

7,58 

0 

.  3.74 

N-2 

2 

3 

2 

19.08 

22.74 

27.84 

26.18 

N-3 

0 

0 

0 

b 

0 

N-4 

0 

0 

0 

0 

0 

0 

0 

1^.  ' 

0 

0 

0 

0 

o' 

N-6 

0 

b 

0 

0 

0 

0 

0 

'N-7 

0 

1 

0 

7.58 

0 

^  ■  3¥4'":.;"'-i 

N-8 

0 

0 

0 

0 

0 

0 

0 

N-9 

r'" 

1 

9.54 

7.58 

7.48  ' 

N-10 

0 

3 

0 

0 

22.74 

0 

11.22 

N-n 

1 

- 

9.54 

15.1b'; 

iii2 

N-12 

0 

0 

0 

0 

0 

0 

0 

N-i3  ' 

0 

0 

0 

0  , 

N-14 

0 

. 0 . 

. .  0 . 

0 

0 

0 

0 

N-15 

,  . 

0 

0 

'  o„;:j 

0 

!'-b 

N-1 6 

0 

0 

0 

0 

1  0 

b 

N-1 7 

0 

b 

'  0 

0 

0 

0 

N-1 8 

0 

0 

i  0 

0 

0 

1  0 

. . . """o . . . . 

N-19 

"■'"O' 

■  ' 

0 

b 

tall! 

i  o'  , 

1  ;  ; 

0 

N-20 

0  " 

0 

0 

0 

. "b . ’ 

1  0 

Note:  N  =  Norfolk;  G  =  Groton 

“n”  =  Sum  of  Symptom  Scores  on  Nausea  Subscale 
“o”  =  Sum  of  Symptom  Scores  on  Oculomotor  Subscale 
“d”  =  Sum  of  Symptom  Scores  on  Disorientation  Subscale 
“N”  =  Weighted  Nausea  Factor  Score  [(n)  x  9.54] 

“O”  =  Weighted  Oculomotor  Factor  Score  [(o)  x  7.58] 

“D”  =  Weighted  Disorientation  Factor  Score  [(d)  x  13.92] 
Total  Score  =  [(n)  +  (o)  +  (d)  x  3.74] 
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APPENDIX  K: 

RESULTS  OF  VESUB  TEE  SIMULATION  SIDE  EFFECTS  ASSESSMENT 

(Continued) 


Transformed  Scores  Prior  to  Orientation  Scenario 


Note:  N  =  Norfolk;  G  =  Groton 

“n”  =  Sum  of  Symptom  Scores  on  Nausea  Subscale 
“o”  =  Sum  of  Symptom  Scores  on  Oculomotor  Subscale 
“d”  =  Sum  of  Symptom  Scores  on  Disorientation  Subscale 
“N”  =  Weighted  Nausea  Factor  Score  [(n)  x  9.54] 

“O”  =  Weighted  Oculomotor  Factor  Score  [(o)  x  7.58] 

“D”  =  Weighted  Disorientation  Factor  Score  [(d)  x  13.92] 
Total  Score  =  [(n)  +  (o)  +  (d)  x  3.74] 
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APPENDIX  K: 

RESULTS  OF  VESUB  TEE  SIMULATION  SIDE  EFFECTS  ASSESSMENT 

(Continued) 


Symptom  Scores  After  Scenarios 


Ss# 

GenDis 

HdAche 

EyeStrn 

DifFoc 

Salivate 

Sweat 

Nausea 

- 

Scenmo 

Scenario 

Scenario 

Scenario 

Scenario 

Scenario 

Scenario 

Scenario 

1 

2 

m. 

1 

m: 

1 

2 

>3^ 

1 

2 

3 

1 

3 

1 

m. 

Ma 

1 

2 

3 

1 

2 

'3' 

N-1 

□ 

0 

0 

0 

0 

0 

0 

0 

0 

0 

b 

b 

a 

b 

b 

0 

b 

b 

b 

b 

b 

b 

N-2  ' 

1 

0 

0 

0 

0 

0 

0 

0 

1 

1 

mA 

b 

b 

b 

b 

b 

b 

b 

0 

0 

b 

b 

N-3 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0  j 

1 

1 

1 

b 

b 

b 

b 

b 

b 

b 

b 

1 

'N-4 

I 

1 

i 

0“ 

1 

0 

0 

1 

1 

7,:- 

\ 

! 

1 

b 

A* 

0 

b 

b 

b 

N-5 

0 

0 

0 

0 

0 

0 

0 

0 

□ 

0 

1 

0 

b 

D 

b 

b 

b 

b 

b 

N-6  r 

0 

0 

0  ^ 

0 

0 

b 

0 

0 

W: 

0 

0 

■qJ 

0 

b 

b 

0 

b 

N-7 

1 

2 

X 

1 

X 

0 

0 

X 

1 

1 

X 

0 

1 

X 

2 

X 

1 

2 

X 

N-8 

0 

1 

1 

0 

0 

b 

b- 

0 

b 

0  ' 

0  : 

b 

b 

b 

b 

0 

M 

b 

M: 

b 

b. 

N-9 

0 

0 

0 

0 

0 

0 

1 

1 

0 

b 

b 

b 

□ 

b 

D 

0 

b 

1 

1 

b 

N-10 

1 

ll 

1 

1 

0 

1 

p:;' 

1 

0 

T~ 

[1 

b 

b"'' 

b 

b 

,0 

ii 

b 

p 

b 

b 

N-11 

1 

0 

0 

0 

0 

0 

0 

El 

0 

0 

b 

b 

D 

b 

b 

b 

b 

b 

b 

N-12 

0 

0 

0 

"11 

0 

b 

0 

0 

0 

b 

0 

o| 

b 

bl 

w- 

0 

hp| 

b 

b 

0 

N-13 

0 

0 

0 

0 

0 

0 

1 

1 

1 

1 

1 

1 

b 

b 

1  b 

b 

b 

N-14 

1 

X- 

1 

2 

X 

i 

X 

;  2 

X 

b 

xl 

1 

1 

■X 

b 

2 

x"" 

N-15 

H 

0 

0 

0 

0 

0 

b 

b 

b 

b 

b 

b 

El 

1 

1 1 

b 

b 

i  b 

N-16  i 

m 

w 

0 

6 

0 

b 

p 

1 

0’ 

III 

b 

H 

b 

:fbi? 

0 

6  : 

;':b' 

b 

b 

b 

N-1 7 

D 

0 

D 

0 

D 

El 

0 

1 

'"’b"'' 

b 

b 

b 

1  b 

'N-18 

0 

m 

0 

HI 

0 

6 

1 

b 

"o 

IP 

b 

0 

b 

b 

N-19 

0 

0 

"o'“ 

Q 

0 

□ 

0 

0 

b 

b 

b 

b 

N-20 

0 

D 

0 

b 

0 

0 

0 

b 

i'  : 

ibi 

b 

b 

b 

Note;  N  =  Norfolk;  G  =  Groton 

0  =  None;  1  =  Slight;  2  =  Moderate;  3  =  Severe 
X  =  Subject  Dropped  from  Evaluation 
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Appendix  K: 

RESULTS  OF  VESUB  TEE  SIMULATION  SIDE  EFFECTS  ASSESSMENT 

(Continued) 


Symptom  Scores  After  Scenarios 


Ss# 

Mm 

HdAche 

EyeStrn 

DilFoc 

Salivate 

Sweat 

Nausea 

Scenario 

1  2  3 

Scenario 

1  2  3 

Scenario 

1  2  3 

Scenario 

1  2  3 

Scenario 

1  2  3 

Scenario 

1  2  3 

Scenario 

1  2  3 

Scenario 

1  2  3 

G-1 

a 

in 

B 

1 

1 

1 

0 

IQ 

0 

B 

1 

2 

0 

1 

1 

0 

B 

0 

0 

0 

0 

0 

0 

G-2 

1 

I 

D 

1 

1 

1 

0 

1 

0 

B 

0 

0 

0 

0 

0 

0 

B 

0 

0 

0 

1 

1 

1 

G-3 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

B 

0 

0 

0 

0 

0 

0 

G-4 

0 

1 

1 

0 

1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

G-5 

0 

2 

1 

2 

2 

2 

1 

2 

1 

1 

1 

1 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

G-6 

0 

0 

0 

0 

1 

1 

0 

0 

0 

0 

1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

G-7 

1 

1 

1 

0 

1 

1 

1 

1 

0 

1 

2 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

G-8 

El 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

B 

0 

0 

0 

0 

0 

0 

0 

0 

0 

G-9 

1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

1 

0 

0 

0 

0 

G-10 

a 

0 

0 

0 

0 

1 

0 

0 

0 

1 

1 

1 

1 

0 

0 

0 

0 

0 

1 

1 

1 

0 

0 

0 

G-11 

0 

1 

0 

0 

0 

0 

0 

1 

1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

2 

1 

0 

1 

1 

G-12 

El 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

G-1 3 

1 

1 

1 

0 

1 

1 

0 

0 

0 

1 

1 

1 

0 

0 

1 

0 

0 

0 

0 

1 

1 

0 

0 

G-1 4 

I 

1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

G-15 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

G-16 

0 

0 

0 

0 

0 

0 

1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

G-1 7 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

il 

0 

0 

0 

0 

0 

0 

0 

G-18 

0 

0 

0 

0 

0 

0 

0 

0 

0 

B 

B 

B 

0 

0 

0 

0 

0 

0 

0 

G-1 9 

0 

0 

0 

0 

0 

1 

1 

1 

B 

B 

n 

0 

1 

1 

0 

1 

0 

0 

0 

G-20 

0 

0 

0 

0 

0 

0 

1 

1 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

1 

1 

0 

0 

G-21 

0 

1 

1 

1 

1 

1 

0 

1 

1 

1 

1 

1 

1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

G-22 

Ell 

0 

0 

1 

1 

1 

0 

0 

0 

1 

1 

0 

! 

ai 

0 

0 

0 

0 

0 

0 

0 

0 

Note:  N  =  Norfolk;  G  =  Groton 

0  =  None;  1  =  Slight;  2  =  Moderate;  3  =  Severe 
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APPENDIX  K: 

RESULTS  OF  VESUB  TEE  SIMULATION  SIDE  EFFECTS  ASSESSMENT 

(Continued) 


Symptom  Scores  After  Scenarios 


Ss# 

DifCon 

FullHd 

BlurVis 

Diz(0) 

Diz(C) 

Vertigo 

Stomch 

Burping 

Scenario 

1  2  3 

Scenario 

1  2  3 

Scenario 

1  2  3 

Scenario 

1  2  3 

Scenario 

1  2  3 

Scenario 
12  3 

*BbibbtibS 

Scenario 

1  2  1 

N-1 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

N-2 

0 

0 

0 

li* 

ii 

1 

0 

'  0*: 

0 

b 

0 

0 

0 

0 

V" 

0 

0 

0 

b~ 

0 

0 

N-3 

0 

0 

0 

0 

0 

0 

1 

0 

1 

1 

1 

1 

1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

N-4 

0 

1 

lil 

;Pli 

0 

0 

0 

0 

r 

1 

m: 

0 

0 

0 

1 

1 

0 

.b'“ 

IS 

0 

0 

0 

0 

N-5 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

N-6 

0 

0 

6 

0 

0 

0 

*0 

0 

W 

bS 

'bS 

0 

0 

0 

0 

0 

0 

b 

0 

N-7 

0 

1 

X 

0 

0 

X 

0 

0 

X 

0 

1 

X 

0 

0 

X 

0 

0 

X 

1 

2 

X 

0 

0 

X 

N-8 

w 

0 

0 

li 

i 

0 

Wf. 

0 

mi 

0 

F 

W 

0 

W] 

b" 

1 

0 

.„;p 

0 

0 

0 

0 

N-9 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

N-10 

6 

0 

0 

1 

S 

0 

I"-' 

1 

1 

'b;:i 

1 

0 

0 

1 

0 

0 

"bs 

0 

1  “ 

0 

0 

N-11 

0 

0 

0 

0 

0 

b 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

-N-1 2 

0 

0 

0 

m 

0 

0 

0 

m 

b 

0 

0 

0 

0 

0 

0 

ll'i 

0 

0 

mi 

1:S 

b 

0 

0 

b 

N-13 

0 

0 

0 

0 

'o'" 

0 

1  ! 

1 

i 

1 

0 

0  1 

0 

b'" 

0 

0  1 

0 

0 

0 

b"i 

0 

6 

0 

0 

0 

N-14 

6 

ii 

X'' 

0 

*0  I 

X 

0 

0 

X 

0 

0 

W' 

0 

b 

X 

0 

0 

X 

1 

X 

0 

•I-  . 

•X 

N-1 5 

0 

0 

0 

0 

'  o""  ! 

0 

1 

0 

'b 

0 

0 

0 

0 

0 

0  1 

0 

0 

0 

0 

0 

"b . 

0 

0 

0 

'N-16 

0 

6 

(O'; 

W. 

0 

0 

IS 

b 

, 

0 

i07 

1*1 

0 

ibS 

0 

0 

"IS 

Wk- 

•o'l 

0 

0 

N-1 7 

0 

0 

0 

0 

'o'"’ 

0 

0 

0 

T 

0 

0 

0 

b“" 

0 

0  1 

b 

0 

0 

6 

'b'" 

0 

0 

0 

0 

N-18 

0 

0 

0 

W 

li 

0 

0 

w. 

0 

0 

0 

0 

0 

i:ii 

li 

si! 

0 

b 

0 

N-1 9 

0 

0 

0 

6 

6“ 

0 

""o . 

b 

b 

0 

"‘"l 

b 

0 

1 

0  j 

“"0“; 

0 

0 

0 

'i'"" 

1 

b 

6 

1 

N-20 

0 

6 

0 

W 

0*: 

0 

if” 

0 

0 

'■b* 

0 

0 

0 

'"bs 

b 

0 

db" 

*bl 

0 

0 

0  1 

0 

Note:  N  =  Norfolk;  G  =  Groton 

0  =  None;  1  =  Slight;  2  =  Moderate;  3  =  Severe 
X  =  Subject  Dropped  from  Evaluation 
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APPENDIX  K: 

RESULTS  OF  VESUB  TEE  SIMULATION  SIDE  EFFECTS  ASSESSMENT 

Continued) 


Symptom  Scores  After  Scenarios 


Ss# 

DifCon 

FullHd 

BlurVis 

Diz(O) 

Diz(C) 

Vertigo 

Stomch 

Burping 

Scenario 

1  2  3 

Scenario 

1  2  3 

Scenario 

1  2  3 

Scenario 

1  2  3 

Scenario 

1  2  3 

Scenario 

1  2  3 

Scenario 

1  2  3 

Scenario 

1  2  3 

G-1 

0 

0 

0 

1 

0 

0 

1 

1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

G-2 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

2 

0 

0 

1 

0 

G-3 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

G-4 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

G-5 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

1 

1 

0 

0 

0 

0 

0 

0 

G-6 

0 

1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

G-1 

0 

0 

0 

1 

1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

2 

1 

0 

0 

0 

G-8 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

G-9 

0 

0 

0 

1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

G-10 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

1 

1 

0 

0 

0 

G-11 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

G-12 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

G-1 3 

0 

0 

0 

1 

1 

1 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

G-14 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

G-15 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

G-1 6 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

G-1 7 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

G-18 

0 

0 

0 

0 

0 

0 

0 

I 

0 

0 

0 

0 

0 

0 

0 

1 

1 

1 

0 

0 

0 

0 

0 

0 

G-1 9 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

1 

0 

0 

0 

0 

G-20 

0 

0 

0 

1 

1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

1 

1 

G-21 

0 

0 

0 

0 

0 

0 

0 

1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

G-22 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Note:  N  =  Norfolk;  G  =  Groton 

0  =  None;  1  =  Slight;  2  =  Moderate;  3  =  Severe 
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APPENDIX  K: 

RESULTS  OF  VESUB  TEE  SIMULATION  SIDE  EFFECTS  ASSESSMENT 

(Continued) 


Transformed  Scores  After  Scenarios 


Ss# 

Tof’n” 

Scenario 

1  2  3 

Tot”o” 
Scenario 
12  3 

Top’d” 

Scenario 

1  2  3 

1 

“N” 

Scenario 

3 

I 

Scenario 

2 

3 

N-1 

0 

0 

1 

0 

D 

1 

0 

1 

0 

0 

0 

9.54 

0 

7.58 

N-2 

Id 

0 

0 

3 

i'. 

1 

3 

0 

0 

0 

0 

22.74 

7.58 

N-3 

0 

0 

1 

2 

2 

1 

4 

4 

4 

0 

0 

9.54 

15.16 

7.58 

'■N-4 

0 

1  '■ 

0 

'3" ' 

4 

3;;' 

3 

1 

VM  : 

9.54 

0 

^.74 

30.32 

37.9 

N-5 

0 

0 

□ 

1 

0 

0 

0 

0 

0 

0 

0 

0 

7.58 

N-6 

0 

i 

0 

W 

El 

9.54 

0 

0 

0 

0 

N-7 

10 

El 

3 

5 

1 

4 

X 

38.16 

95.4 

0 

22.74 

37.9 

0 

N-8 

0 

El 

1” 

ss 

0 

0 

7.58 

7.58 

N-9 

1 

0 

1 

1 

0 

1 

1 

0 

9.54 

9.54 

0 

7.58 

7.58 

0 

N-10 

0 

0 

4^ 

■3 

3 

5 

2 

a:-k 

0 

'  '  0' 

30.32 

22.74 

22.74 

N-11 

0 

D 

1 

0 

□ 

0 

0 

0 

0 

0 

0 

i  7.58 

N-12 

0 

SI 

0 

0 

0 

6 

b 

N-1 3 

El 

0 

D 

3 

2  : 

2 

“2 

0 

. . '0" 

0 

22.74 

22.74 

22.74 

•■N-14 

2 

6 

0 

5 

Hi 

2 

3~ 

S'-" 

19.08 

0 

37.9 

45.48 

0 

N-1 5 

0 

1 

1 

1 

0 

1 

El 

0 

0 

9.54 

9.54 

7.58 

0 

0 

N-16 

0 

6 

0 

U:' 

■4,:?: 

S' 

0 

. 0 

■  ■  7.58 

:'X-1S:16 

N-17 

0 

0 

□ 

1 

0 

0 

1 

El 

'o'"'’ 

0 

0 

0 

7.58 

0 

0 

N-1 8 

0 

0 

■1'^, 

2 

0 

0 

0 

0 

0 

'  15.16 

'O': 

N-1 9 

0 

1 

2 

0 

1 

0 

2 

0 

0 

.  9.54 

19.08 

0 

7.58 

0 

N-20 

1 

.0'.s 

El 

v2 

El 

m 

of 

0 

0 

0 

. '  ■  0 

15.16 

b 

0 

Note:  N  =  Norfolk;  G  =  Groton 

“n”  =  Sum  of  Symptom  Scores  on  Nausea  Subscale 
“o”  =  Sum  of  Symptom  Scores  on  Oculomotor  Subscale 
*  “d”  =  Sum  of  Symptom  Scores  on  Disorientation  Subscale 

“N”  =  Weighted  Nausea  Factor  Score  [(n)  x  9.54] 

^  “0”  =  Weighted  Oculomotor  Factor  Score  [(o)  x  7.58] 

“D”  =  Weighted  Disorientation  Factor  Score  [(d)  x  13.92] 
Total  Score  =  [(n)  +  (o)  +  (d)  X  3.74] 

X  =  Subject  Dropped  from  Evaluation 
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APPENDIX  K: 

RESULTS  OF  VESUB  TEE  SIMULATION  SIDE  EFFECTS  ASSESSMENT 

(Continued) 


Transformed  Scores  After  Scenarios 


Ss# 

Tot”n” 

Tot”o” 

Tot”d” 

“N” 

”0” 

Scenario 

Scenario 

Scenario 

Scenario 

Scenario 

1 

2 

3 

1 

2 

3 

1 

2 

3 

1 

2 

3 

1 

2 

3 

G-1 

a 

0 

m 

4 

5 

6 

2 

2 

2 

0 

0 

0 

30.32 

37.9 

45.48 

G-2 

4 

1 

3 

3 

1 

1 

1 

1 

9.54 

38.16 

9.54 

22.74 

22.74 

7.58 

G-3 

0 

□ 

D 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

G-4 

1 

0 

2 

2 

0 

0 

0 

0 

9.54 

0 

0 

15.16 

15.16 

G-5 

0 

□ 

4 

7 

7 

0 

1 

3 

0 

0 

0 

30.32 

53.06 

53.06 

G-6 

1 

1 

0 

3 

3 

0 

0 

0 

0 

9.54 

9.54 

0 

22.74 

22.74 

G-7 

1 

3 

1 

3 

5 

2 

1 

1 

1 

9.54 

28.62 

9.54 

22.74 

37.9 

15.16 

G-8 

0 

0 

0 

a 

D 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

G-9 

1 

1 

El 

1 

1 

□ 

1 

1 

0 

9.54 

9.54 

0 

7.58 

7.58 

0 

G-10 

2 

2 

2 

3 

1 

2 

2 

0 

0 

19.08 

19.08 

19.08 

22.74 

7.58 

15.16 

G-11 

0 

4 

2 

1 

3 

1 

0 

2 

2 

0 

38.16 

19.08 

7.58 

22.74 

7.58 

G-12 

0 

1 

El 

a 

0 

El 

0 

0 

0 

0 

9.54 

0 

0 

0 

0 

G-13 

1 

1 

1 

2 

3 

5 

1 

1 

3 

9.54 

9.54 

9.54 

15.16 

22.74 

37.9 

G-14 

D 

2 

H 

1 

1 

1 

0 

2 

0 

0 

19.08 

0 

7.58 

7.58 

7.58 

G-1 5 

0 

0 

D 

m 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

7.58 

G-16 

a 

0 

0 

1 

1 

0 

0 

0 

0 

0 

0 

0 

7.58 

7.58 

G-1 7 

0 

0 

□ 

El 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

G-1 8 

0 

m 

H 

0 

3 

0 

1 

3 

1 

0 

0 

0 

0 

22.74 

0 

G-19 

2 

3 

D 

2 

1 

1 

0 

0 

0 

19.08 

28.62 

0 

15.16 

7.58 

7.58 

G-20 

1 

2 

2 

El 

1 

2 

1 

1 

1 

9.54 

19.08 

19.08 

0 

7.58 

15.16 

G-21 

0 

0 

0 

3 

5 

6 

1 

1 

2 

0 

0 

0 

22.74 

37.9 

45.48 

G-22 

0 

0 

m 

3 

3 

1 

1 

1 

0 

0 

0 

0 

22.74 

22.74 

7.58 

Note:  N  =  Norfolk;  G  =  Groton 

“n”  =  Sum  of  Symptom  Scores  on  Nausea  Subscale  * 

“o”  =  Sxun  of  Symptom  Scores  on  Oculomotor  Subscale 

“d”  =  Sum  of  Symptom  Scores  on  Disorientation  Subscale  „ 

“N”  =  Weighted  Nausea  Factor  Score  [(n)  x  9.54] 

“0”  =  Weighted  Oculomotor  Factor  Score  [(o)  x  7.58] 

“D”  =  Weighted  Disorientation  Factor  Score  [(d)  x  1 3.92] 

Total  Score  =  [(n)  +  (o)  +  (d)  x  3.74] 
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APPENDIX  K: 

RESULTS  OF  VESUB  TEE  SIMULATION  SIDE  EFFECTS  ASSESSMENT 

(Continued) 


Transformed  Scores  After  Scenarios  and  Time  in  HMD 


Ss# 

”D” 

Total  Score 

Duration  (Min.Sec) 

1 

Scenario 

2 

3 

Scenario 
1  2 

I 

Scenario 

2 

N-1 

0 

13.92 

0 

0 

3.74 

7.48 

20.28 

47.03 

34.29 

N-2 

41.76 

0 

0 

22.44 

"  3.74 

'  3.74 

27.40 

44.39 

39.10 

N-3 

55.68 

55.68 

55.68 

22.44 

22.44 

22.44 

22.10 

37.49 

31.43 

N-4 

41.76 

41.76 

"13.92 

22.44 

29.92 

22.44 

19.52 

‘47.04 

N-5 

0 

0 

0 

0 

0 

3M 

19.00 

35.48 

40.05 

N-6 

0 

0 

3.74 

0 

23.05 

42.47 

39.24 

N-7 

13.92 

55.68 

X 

29.92 

71.06 

X 

22.23 

X 

N-8 

0 

13.92 

13.92 

7.48 

7.48 

42.02 

N-9 

13.92 

13.92 

0 

11.22 

11.22 

0 

20.19 

50.32 

N-IO 

69.60 

27.84 

13.92 

':::37.4 

18.70 

:  ■;Jl4.9f'' 

22.38 

29.32 

N-11 

0  1 

0  1 

0 

0 

0 

3.74  ■ 

27.02 

54.47 

40.55 

N-12 

0 

0  ; 

0 

.  d 

-  -  23.11 

■piipiii 

iHBlI 

N-13 

27.84 

27.84 

27.84 

1  '  18.70 

18.70 

’^*‘18.7 

8.39 

53.25 

39.34 

N-14 

27.84 

41.76 

X 

33.66 

56.10 

''  X 

20.52 

N-15 

13.92 

0 

0 

1  7.48 

3.74 

3.74 

20.47 

33.36 

N-1 6 

d 

0 

0 

3.74 

'  3  >4 ' 

'  ■■  7.48 

19.38 

53.58 

21.25 

N-1 7 

13.92 

0 

i  0 

7.48 

0 

i  0 

18.35 

51.31 

33.03 

N-18 

0 

0 

0 

3.74 

7.48 

"o' 

25^00 

. . ■"47.52 

N-19 

0 

27.84 

0 

0 

14.96 

7.48 

16.58 

37.58 

N-20 

27.84 

0 

■;  ,  0 

14.96 

6 

■  20.51 

44.50 

Note:  N  =  Norfolk;  G  =  Groton 

“n”  =  Sum  of  Symptom  Scores  on  Nausea  Subscale 
“o”  =  Sum  of  Symptom  Scores  on  Oculomotor  Subscale 
“d”  =  S\un  of  Symptom  Scores  on  Disorientation  Subscale 
“N”  =  Weighted  Nausea  Factor  Score  [(n)  x  9.54] 

“O”  =  Weighted  Oculomotor  Factor  Score  [(o)  x  7.58] 

“D”  =  Weighted  Disorientation  Factor  Score  [(d)  x  13.92] 

Total  Score  =  [(n)  +  (o)  +  (d)  x  3.74] 

X=  Subject  Dropped  from  Evaluation;  (*)  =  Time  Subject  Was  Removed 


K-13 


Technical  Report  98-003 


APPENDIX  K: 

RESULTS  OF  VESUB  TEE  SIMULATION  SIDE  EFFECTS  ASSESSMENT 

(Continued) 


Transformed  Scores  After  Scenarios  and  Time  in  HMD 


Ss# 

”D” 

Total  Score 

1 

Scenario 

2 

3 

Scenario 
1  2 

3 

1 

Scenario 

2 

3 

G-1 

27.84 

27.84 

22.44 

26.18 

29.92 

20.38 

51.25 

39.48 

G-2 

0 

13.92 

13.92 

3.74 

29.92 

11.22 

21.51 

49.48 

36.39 

G-3 

0 

0 

0 

0 

0 

0 

23.47 

50.47 

36.56 

G-4 

0 

0 

0 

0 

11.22 

7.48 

32.39 

54.57 

37.42 

G-5 

13.92 

13.92 

41.76 

14.96 

29.92 

37.4 

25.41 

59.55 

G-6 

0 

0 

0 

0 

14.96 

14.96 

19.06 

57.32 

43.46 

G-7 

13.92 

13.92 

13.92 

14.96 

14.96 

28.40 

36.38 

G-8 

0 

0 

0 

0 

0 

0 

19.36 

39.03 

33.36 

G-9 

13.92 

13.92 

0 

7.48 

11.22 

0 

20.19 

33.58 

G-10 

0 

0 

0 

11.22 

11.22 

14.96 

18.47 

39.21 

G-11 

0 

27.84 

27.84 

0 

33.66 

18.7 

25.04 

54.48 

43.17 

G-12 

13.92 

0 

0 

7.48 

3.74 

0 

20.45 

G-1 3 

13.92 

13.92 

41.76 

7.48 

18.7 

33.66 

22.58 

54.54 

G-14 

0 

27.84 

0 

3.74 

18.7 

3.74 

35.19 

54.30 

42.31 

G-1 5 

0 

0 

0 

3.74 

0 

3.74 

27.39 

38.14 

G-1 6 

0 

0 

0 

0 

3.74 

3.74 

18.19 

46.47 

33.28 

G-1 7 

0 

0 

0 

0 

0 

0 

31.00 

35.36 

G-1 8 

0 

41.76 

13.92 

0 

22.44 

3.74 

18.57 

53.36 

39.22 

G-1 9 

0 

0 

0 

11.22 

14.96 

3.74 

21.48 

51.55 

G-20 

27.84 

13.92 

13.92 

14.96 

14.96 

18.7 

21.42 

34.25 

G-21 

0 

13.92 

27.84 

7.48 

22.44 

29.92 

18.01 

42.49 

34.39 

G-22 

0 

13.92 

0 

3.74 

14.96 

3.74 

18.00 

49.29 

Note:  N  =  Norfolk;  G  =  Groton 

“n”  =  Sum  of  Symptom  Scores  on  Nausea  Subscale 
“o”  =  Sum  of  Symptom  Scores  on  Oculomotor  Subscale 
“d”  =  Sum  of  Symptom  Scores  on  Disorientation  Subscale 
“N”  =  Weighted  Nausea  Factor  Score  [(n)  x  9.54] 

“0”  =  Weighted  Oculomotor  Factor  Score  [(o)  x  7.58] 

“D”  =  Weighted  Disorientation  Factor  Score  [(d)  x  13.92] 
Total  Score  =  [(n)  +  (o)  +  (d)  x  3.74] 
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